Explore every episode of BSD Now
Dive into the complete episode list for BSD Now. Each episode is cataloged with detailed descriptions, making it easy to find and explore specific topics. Keep track of all episodes from your favorite podcast and never miss a moment of insightful content.
| Pub. Date | Title | Duration | |
|---|---|---|---|
| 08 Nov 2017 | 219: We love the ARC | 02:10:29 | |
Papers we love: ARC by Bryan Cantrill, SSD caching adventures with ZFS, OpenBSD full disk encryption setup, and a Perl5 Slack Syslog BSD daemon.
This episode was brought to you by
  
Headlines
Papers We Love: ARC: A Self-Tuning, Low Overhead Replacement Cache (https://www.youtube.com/watch?v=F8sZRBdmqc0&feature=youtu.be)
Ever wondered how the ZFS ARC (Adaptive Replacement Cache) works?
How about if Bryan Cantrill presented the original paper on its design?
Today is that day.
Slides (https://www.slideshare.net/bcantrill/papers-we-love-arc-after-dark)
It starts by looking back at a fundamental paper from the 40s where the architecture of general-purpose computers are first laid out
The main is the description of memory hierarchies, where you have a small amount of very fast memory, then the next level is slower but larger, and on and on. As we look at the various L1, L2, and L3 caches on a CPU, then RAM, then flash, then spinning disks, this still holds true today.
The paper then does a survey of the existing caching policies and tries to explain the issues with each. This includes ‘MIN’, which is the theoretically optimal policy, which requires future knowledge, but is useful for setting the upper bound, what is the best we could possibly do.
The paper ends up showing that the ARC can end up being better than manually trying to pick the best number for the workload, because it adapts as the workload changes
At about 1:25 into the video, Bryan start talking about the practical implementation of the ARC in ZFS, and some challenges they have run into recently at Joyent.
A great discussion about some of the problems when ZFS needs to shrink the ARC. Not all of it applies 1:1 to FreeBSD because the kernel and the kmem implementation are different in a number of ways
There were some interesting questions asked at the end as well
***
How do I use man pages to learn how to use commands? (https://unix.stackexchange.com/a/193837)
nwildner on StackExchange has a very thorough answer to the question how to interpret man pages to understand complicated commands (xargs in this case, but not specifically).
Have in mind what you want to do.
When doing your research about xargs you did it for a purpose, right? You had a specific need that was reading standard output and executing commands based on that output.
But, when I don't know which command I want?
Use man -k or apropos (they are equivalent). If I don't know how to find a file: man -k file | grep search. Read the descriptions and find one that will better fit your needs.
Apropos works with regular expressions by default, (man apropos, read the description and find out what -r does), and on this example I'm looking for every manpage where the description starts with "report".
Always read the DESCRIPTION before starting
Take a time and read the description. By just reading the description of the xargs command we will learn that:
xargs reads from STDIN and executes the command needed. This also means that you will need to have some knowledge of how standard input works, and how to manipulate it through pipes to chain commands
The default behavior is to act like /bin/echo. This gives you a little tip that if you need to chain more than one xargs, you don't need to use echo to print.
We have also learned that unix filenames can contain blank and newlines, that this could be a problem and the argument -0 is a way to prevent things explode by using null character separators. The description warns you that the command being used as input needs to support this feature too, and that GNU find support it. Great. We use a lot of find with xargs.
xargs will stop if exit status 255 is reached.
Some descriptions are very short and that is generally because the software works on a very simple way. Don't even think of skipping this part of the manpage ;)
Other things to pay attention...
You know that you can search for files using find. There is a ton of options and if you only look at the SYNOPSIS, you will get overwhelmed by those. It's just the tip of the iceberg. Excluding NAME, SYNOPSIS, and DESCRIPTION, you will have the following sections:
When this method will not work so well...
+ Tips that apply to all commands
Some options, mnemonics and "syntax style" travel through all commands making you buy some time by not having to open the manpage at all. Those are learned by practice and the most common are:
Generally, -v means verbose. -vvv is a variation "very very verbose" on some software.
Following the POSIX standard, generally one dash arguments can be stacked. Example: tar -xzvf, cp -Rv.
Generally -R and/or -r means recursive.
Almost all commands have a brief help with the --help option.
--version shows the version of a software.
-p, on copy or move utilities means "preserve permissions".
-y means YES, or "proceed without confirmation" in most cases.
Default values of commands.
At the pager chunk of this answer, we saw that less -is is the pager of man. The default behavior of commands are not always shown at a separated section on manpages, or at the section that is most top placed.
You will have to read the options to find out defaults, or if you are lucky, typing /pager will lead you to that info. This also requires you to know the concept of the pager(software that scrolls the manpage), and this is a thing you will only acquire after reading lots of manpages.
And what about the SYNOPSIS syntax?
After getting all the information needed to execute the command, you can combine options, option-arguments and operands inline to make your job done. Overview of concepts:
Options are the switches that dictates a command behavior. "Do this" "don't do this" or "act this way". Often called switches.
Check out the full answer and see if it helps you better grasp the meaning of a man page and thus the command.
***
My adventure into SSD caching with ZFS (Home NAS) (https://robertputt.co.uk/my-adventure-into-ssd-caching-with-zfs-home-nas.html)
Robert Putt as written about his adventure using SSDs for caching with ZFS on his home NAS.
Recently I decided to throw away my old defunct 2009 MacBook Pro which was rotting in my cupboard and I decided to retrieve the only useful part before doing so, the 80GB Intel SSD I had installed a few years earlier. Initially I thought about simply adding it to my desktop as a bit of extra space but in 2017 80GB really wasn’t worth it and then I had a brainwave… Lets see if we can squeeze some additional performance out of my HP Microserver Gen8 NAS running ZFS by installing it as a cache disk.
I installed the SSD to the cdrom tray of the Microserver using a floppy disk power to SATA power converter and a SATA cable, unfortunately it seems the CD ROM SATA port on the motherboard is only a 3gbps port although this didn’t matter so much as it was an older 3gbps SSD anyway. Next I booted up the machine and to my suprise the disk was not found in my FreeBSD install, then I realised that the SATA port for the CD drive is actually provided by the RAID controller, so I rebooted into intelligent provisioning and added an additional RAID0 array with just the 1 disk to act as my cache, in fact all of the disks in this machine are individual RAID0 arrays so it looks like just a bunch of disks (JBOD) as ZFS offers additional functionality over normal RAID (mainly scrubbing, deduplication and compression).
Configuration
Lets have a look at the zpool before adding the cache drive to make sure there are no errors or uglyness:
Now lets prep the drive for use in the zpool using gpart. I want to split the SSD into two seperate partitions, one for L2ARC (read caching) and one for ZIL (write caching). I have decided to split the disk into 20GB for ZIL and 50GB for L2ARC. Be warned using 1 SSD like this is considered unsafe because it is a single point of failure in terms of delayed writes (a redundant configuration with 2 SSDs would be more appropriate) and the heavy write cycles on the SSD from the ZIL is likely to kill it over time.
Now it’s time to see if adding the cache has made much of a difference. I suspect not as my Home NAS sucks, it is a HP Microserver Gen8 with the crappy Celeron CPU and only 4GB RAM, anyway, lets test it and find out. First off lets throw fio at the mount point for this zpool and see what happens both with the ZIL and L2ARC enabled and disabled.
Observations
Ok, so the initial result is a little dissapointing, but hardly unexpected, my NAS sucks and there are lots of bottle necks, CPU, memory and the fact only 2 of the SATA ports are 6gbps. There is no real difference performance wise in comparison between the results, the IOPS, bandwidth and latency appear very similar. However lets bare in mind fio is a pretty hardcore disk benchmark utility, how about some real world use cases?
Next I decided to test a few typical file transactions that this NAS is used for, Samba shares to my workstation. For the first test I wanted to test reading a 3GB file over the network with both the cache enabled and disabled, I would run this multiple times to ensure the data is hot in the L2ARC and to ensure the test is somewhat repeatable, the network itself is an uncongested 1gbit link and I am copying onto the secondary SSD in my workstation. The dataset for these tests has compression and deduplication disabled.
Samba Read Test
Not bad once the data becomes hot in the L2ARC cache reads appear to gain a decent advantage compared to reading from the disk directly. How does it perform when writing the same file back accross the network using the ZIL vs no ZIL.
Samba Write Test
Another good result in the real world test, this certainately helps the write transfer speed however I do wonder what would happen if you filled the ZIL transferring a very large file, however this is unlikely with my use case as I typically only deal with a couple of files of several hundred megabytes at any given time so a 20GB ZIL should suit me reasonably well.
Is ZIL and L2ARC worth it?
I would imagine with a big beefy ZFS server running in a company somewhere with a large disk pool and lots of users with multiple enterprise level SSD ZIL and L2ARC would be well worth the investment, however at home I am not so sure. Yes I did see an increase in read speeds with cached data and a general increase in write speeds however it is use case dependant. In my use case I rarely access the same file frequently, my NAS primarily serves as a backup and for archived data, and although the write speeds are cool I am not sure its a deal breaker. If I built a new home NAS today I’d probably concentrate the budget on a better CPU, more RAM (for ARC cache) and more disks. However if I had a use case where I frequently accessed the same files and needed to do so in a faster fashion then yes, I’d probably invest in an SSD for caching. I think if you have a spare SSD lying around and you want something fun todo with it, sure chuck it in your ZFS based NAS as a cache mechanism. If you were planning on buying an SSD for caching then I’d really consider your needs and decide if the money can be spent on alternative stuff which would improve your experience with your NAS. I know my NAS would benefit more from an extra stick of RAM and a more powerful CPU, but as a quick evening project with some parts I had hanging around adding some SSD cache was worth a go.
More Viewer Interview Questions for Allan
News Roundup
Setup OpenBSD 6.2 with Full Disk Encryption (https://blog.cagedmonster.net/setup-openbsd-with-full-disk-encryption/)
Here is a quick way to setup (in 7 steps) OpenBSD 6.2 with the encryption of the filesystem.
First step: Boot and start the installation:
(I)nstall: I
Keyboard Layout: ENTER (I'm french so in my case I took the FR layout)
Leave the installer with: !
Second step: Prepare your disk for encryption.
Using a SSD, my disk is named : sd0, the name may vary, for example : wd0.
Initiating the disk:
Configure your volume:
Now we'll use bioctl to encrypt the partition we created, in this case : sd0a (disk sd0 + partition « a »).
Enter your passphrase.
Third step:
Let's resume the OpenBSD's installer. We follow the install procedure
Fourth step: Partitioning of the encrypted volume.
We select our new volume, in this case: sd1
The whole disk will be used: W(hole)
Let's create our partitions:
NB: You are more than welcome to create multiple partitions for your system.
Fifth step: System installation
It's time to choose how we'll install our system (network install by http in my case)
Sixth step: Finalize the installation.
Last step: Reboot and start your system.
Put your passphrase. Welcome to OpenBSD 6.2 with a full encrypted file system.
Optional: Disable the swap encryption.
The swap is actually part of the encrypted filesystem, we don't need OpenBSD to encrypt it. Sysctl is giving us this possibility.
Step-by-Step FreeBSD installation with ZFS and Full Disk Encryption (https://blog.cagedmonster.net/step-by-step-freebsd-installation-with-full-disk-encryption/)
1. What do I need?
For this tutorial, the installation has been made on a Intel Core i7 - AMD64 architecture.
On a USB key, you would probably use this link : ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-mini-memstick.img
If you can't do a network installation, you'd better use this image : ftp://ftp.freebsd.org/pub/FreeBSD/releases/amd64/amd64/ISO-IMAGES/11.1/FreeBSD-11.1-RELEASE-amd64-memstick.img
You can write the image file on your USB device (replace XXXX with the name of your device) using dd : # dd if=FreeBSD-11.1-RELEASE-amd64-mini-memstick.img of=/dev/XXXX bs=1m
2. Boot and install: Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F1.png)
3. Configure your keyboard layout: Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F2.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F3.png)
4. Hostname and system components configuration :
Set the name of your machine: [Screenshot](https://blog.cagedmonster.net/content/images/2017/09/F4.png_
What components do you want to install? Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F5.png)
5. Network configuration:
Select the network interface you want to configure. Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F6.png)
First, we configure our IPv4 network. I used a static adress so you can see how it works, but you can use DHCP for an automated configuration, it depends of what you want to do with your system (desktop/server) Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F7.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F7-1.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F8.png)
IPv6 network configuration. Same as for IPv4, you can use SLAAC for an automated configuration. Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F9.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F10-1.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F10-2.png)
Here, you can configure your DNS servers, I used the Google DNS servers so you can use them too if needed. Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F11.png)
6. Select the server you want to use for the installation:
I always use the IPv6 mirror to ensure that my IPv6 network configuration is good.Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F12.png)
7. Disk configuration:
As we want to do an easy full disk encryption, we'll use ZFS. Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F13.png)
Make sure to select the disk encryption :Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F14.png)
Launch the disk configuration :Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F15.png)
Here everything is normal, you have to select the disk you'll use :Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F16.png)
I have only one SSD disk named da0 :Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F17.png)
Last chance before erasing your disk :Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F18.png)
Time to choose the password you'll use to start your system : Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F19.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F20.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F21.png)
8. Last steps to finish the installation:
The installer will download what you need and what you selected previously (ports, src, etc.) to create your system: Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F22.png)
8.1. Root password:
Enter your root password: Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F22-1.png)
8.2. Time and date:
Set your timezone, in my case: Europe/France Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F22-2.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F23.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F23-1.png)
Make sure the date and time are good, or you can change them :Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F24.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F25.png)
8.3. Services:
Select the services you'll use at system startup depending again of what you want to do. In many cases powerd and ntpd will be useful, sshd if you're planning on using FreeBSD as a server. Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F26.png)
8.4. Security:
Security options you want to enable. You'll still be able to change them after the installation with sysctl. Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F26-1.png)
8.5. Additionnal user:
Create an unprivileged system user: Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F26-2.png)
Make sure your user is in the wheel group so he can use the su command. Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F26-3.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F26-4.png)
8.6. The end:
End of your configuration, you can still do some modifications if you want : Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F26-5.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F26-6.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F26-7.png)
9. First boot:
Enter the passphrase you have chosen previously : Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F27.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F28.png) & Screenshot (https://blog.cagedmonster.net/content/images/2017/09/F29.png)
Welcome to Freebsd 11.1 with full disk encryption!
***
The anatomy of ldd program on OpenBSD (http://nanxiao.me/en/the-anatomy-of-ldd-program-on-openbsd/)
In the past week, I read the ldd (https://github.com/openbsd/src/blob/master/libexec/ld.so/ldd/ldd.c) source code on OpenBSD to get a better understanding of how it works. And this post should also be a reference for other*NIX OSs.
The ELF (https://en.wikipedia.org/wiki/Executable_and_Linkable_Format) file is divided into 4 categories: relocatable, executable, shared, and core. Only the executable and shared object files may have dynamic object dependencies, so the ldd only check these 2 kinds of ELF file:
(1) Executable. ldd leverages the LD_TRACE_LOADED_OBJECTS environment variable in fact, and the code is as following:
if (setenv("LD_TRACE_LOADED_OBJECTS", "true", 1) < 0)
err(1, "setenv(LD_TRACE_LOADED_OBJECTS)");
When LDTRACELOADED_OBJECTS is set to 1 or true, running executable file will show shared objects needed instead of running it, so you even not needldd to check executable file. See the following outputs:
$ /usr/bin/ldd
usage: ldd program ...
$ LD_TRACE_LOADED_OBJECTS=1 /usr/bin/ldd
Start End Type Open Ref GrpRef Name
00000b6ac6e00000 00000b6ac7003000 exe 1 0 0 /usr/bin/ldd
00000b6dbc96c000 00000b6dbcc38000 rlib 0 1 0 /usr/lib/libc.so.89.3
00000b6d6ad00000 00000b6d6ad00000 rtld 0 1 0 /usr/libexec/ld.so
(2) Shared object. The code to print dependencies of shared object is as following:
if (ehdr.e_type == ET_DYN && !interp) {
if (realpath(name, buf) == NULL) {
printf("realpath(%s): %s", name,
strerror(errno));
fflush(stdout);
_exit(1);
}
dlhandle = dlopen(buf, RTLD_TRACE);
if (dlhandle == NULL) {
printf("%s\n", dlerror());
fflush(stdout);
_exit(1);
}
_exit(0);
}
Why the condition of checking a ELF file is shared object or not is like this:
if (ehdr.e_type == ET_DYN && !interp) {
......
}
That’s because the file type of position-independent executable (PIE) is the same as shared object, but normally PIE contains a interpreter program header since it needs dynamic linker to load it while shared object lacks (refer this article). So the above condition will filter PIE file.
The dlopen(buf, RTLD_TRACE) is used to print dynamic object information. And the actual code is like this:
if (_dl_traceld) {
_dl_show_objects();
_dl_unload_shlib(object);
_dl_exit(0);
}
In fact, you can also implement a simple application which outputs dynamic object information for shared object yourself:
# include | |||
| 11 Jun 2014 | 41: Commit This Bit | 01:07:04 | |
This week in the big show, we'll be interviewing Benedict Reuschling of the FreeBSD documentation team, and he has a special surprise in store for Allan. As always, answers to your questions and all the latest news, on BSD Now - the place to B.. SD.
This episode was brought to you by
 
Headlines
FreeBSD moves to Bugzilla (https://lists.freebsd.org/pipermail/freebsd-announce/2014-June/001559.html)
Historically, FreeBSD has used the old GNATS system for keeping track of bug reports
After years and years of wanting to switch, they've finally moved away from GNATS to Bugzilla
It offers a lot of advantages, is much more modern and actively maintained and
There's a new workflow chart (http://people.freebsd.org/~eadler/bugrelocation/workflow.html) for developers to illustrate the new way of doing things
The old "send-pr" command will still work for the time being, but will eventually be phased out in favor of native Bugzilla reporting tools (of which there are multiple in ports)
This will hopefully make reporting bugs a lot less painful
***
DIY NAS: EconoNAS 2014 (http://blog.brianmoses.net/2014/06/diy-nas-econonas-2014.html)
We previously covered this blog last year, but the 2014 edition is up
More of a hardware-focused article, the author details the parts he's using for a budget NAS
Details the motherboard, RAM, CPU, hard drives, case, etc
With a set goal of $500 max, he goes just over it - $550 for all the parts
Lots of nice pictures of the hardware and step by step instructions for assembly, as well as software configuration instructions
***
DragonflyBSD 3.8 released (http://www.shiningsilence.com/dbsdlog/2014/06/04/14122.html)
Justin (http://www.bsdnow.tv/episodes/2013_11_13-the_gateway_drug) announced the availability of DragonflyBSD 3.8.0
Binaries in /bin and /sbin are dynamic now, enabling the use of PAM and NSS to manage user accounts
It includes a new HAMMER FS backup script and lots of FreeBSD tools have been synced with their latest versions
Work continues on for the Intel graphics drivers, but it's currently limited to the HD4000 and Ivy Bridge series
See the release page (http://www.dragonflybsd.org/release38/) for more info and check the link for source-based upgrade instructions
***
OpenZFS European conference 2014 (http://www.open-zfs.org/wiki/Publications#2014_OpenZFS_European_Conference)
There was an OpenZFS conference held in Europe recently, and now the videos are online for your viewing pleasure
Matt Ahrens, Introduction (http://www.youtube.com/watch?v=Mk1czZs6vkQ)
Michael Alexander, FhGFS performance on ZFS (http://www.youtube.com/watch?v=Ak1HB507-xY)
Andriy Gapon, Testing ZFS on FreeBSD (http://www.youtube.com/watch?v=oB-QDwVuBH4)
Luke Marsden, HybridCluster: ZFS in the cloud (http://www.youtube.com/watch?v=ISI9Ppj3kTo)
Vadim Comănescu, Syneto: continuously delivering a ZFS-based OS (http://www.youtube.com/watch?v=1xK94v0BedE)
Chris George, DDRdrive ZIL accelerator: random write revelation (http://www.youtube.com/watch?v=ScNHjWBQYQ8)
Grenville Whelan, High-Availability (http://www.youtube.com/watch?v=tiTYZykCeDo)
Phil Harman, Harman Holistic (https://www.youtube.com/watch?v=ApjkrBVlPXk)
Mark Rees, Storiant and OpenZFS (http://www.youtube.com/watch?v=41yl23EACns)
Andrew Holway, EraStor ZFS appliances (http://www.youtube.com/watch?v=b4L0DRvKJxo)
Dan Vâtca, Syneto and OpenZFS (http://www.youtube.com/watch?v=pPOW8bwUXxo)
Luke Marsden, HybridCluster and OpenZFS (http://www.youtube.com/watch?v=uSM1s1aWlZE)
Matt Ahrens, Delphix and OpenZFS (http://www.youtube.com/watch?v=UaRdzUOsieA)
Check the link for slides and other goodies
***
Interview - Benedict Reuschling - bcr@freebsd.org (mailto:bcr@freebsd.org)
BSD documentation, getting commit access, unix education, various topics
News Roundup
Getting to know your portmgr, Steve Wills (http://blogs.freebsdish.org/portmgr/2014/06/04/getting-to-know-your-portmgr-steve-wills/)
"It is my pleasure to introduce Steve Wills, the newest member of the portmgr team"
swills is an all-round good guy, does a lot for ports (especially the ruby ports)
In this interview, we learn why he uses FreeBSD, the most embarrassing moment in his FreeBSD career and much more
He used to work for Red Hat, woah
***
BSDTalk episode 242 (http://bsdtalk.blogspot.com/2014/06/bsdtalk242-pfsense-with-chris-buechler.html)
This time on BSDTalk, Will interviews Chris Buechler (http://www.bsdnow.tv/episodes/2014_02_19-a_sixth_pfsense) from pfSense
Topics include: the heartbleed vulnerability and how it affected pfSense, how people usually leave their firewalls unpatched for a long time (or even forget about them!), changes between major versions, the upgrade process, upcoming features in their 10-based version, backporting drivers and security fixes
They also touch on recent concerns in the pfSense community about their license change, that they may be "going commercial" and closing the source - so tune in to find out what their future plans are for all of that
***
Turn old PC hardware into a killer home server (http://www.pcworld.com/article/2243748/turn-old-pc-hardware-into-a-killer-home-server-with-freenas.html)
Lots of us have old hardware lying around doing nothing but collecting dust
Why not turn that old box into a modern file server with FreeNAS and ZFS?
This article goes through the process of setting up a NAS, gives a little history behind the project and highlights some of the different protocols FreeNAS can use (NFS, SMB, AFS, etc)
Most of our users are already familiar with all of this stuff, nothing too advanced
Good to see BSD getting some well-deserved attention on a big mainstream site
***
Unbloating the VAX install CD (https://blog.netbsd.org/tnf/entry/unbloating_the_vax_install_cd)
After a discussion on the VAX mailing list, something very important came to the attention of the developers...
You can't boot NetBSD on a VAX box with 16MB of RAM from the CD image
This blog post goes through the developer's adventure in trying to fix that through emulation and stripping various things out of the kernel to make it smaller
In the end, he got it booting - and now all three VAX users who want to run NetBSD can do so on their systems with 16MB of RAM...
***
Feedback/Questions
Thomas writes in (http://slexy.org/view/s211mNScBr)
Reynold writes in (http://slexy.org/view/s21JA8BVmZ)
Bostjan writes in (http://slexy.org/view/s2kwS3ncTY)
Paul writes in (http://slexy.org/view/s2VgjXUfW9)
John writes in (http://slexy.org/view/s202AAQUXt)
***
| |||
| 13 Jul 2016 | 150: Sprinkle a little BSD into your life. | 01:21:34 | |
Today on the show, we are going to be talking to Jim Brown (of BSD Cert Fame) about his home-brew sprinkler system… Wait for it…
This episode was brought to you by
Headlines
Distrowatch reviews OpenBSD and PCBSD's live upgrade method (http://distrowatch.com/weekly.php?issue=20160620#upgrade)
Upgrading… The bane of any sysadmin! Distrowatch has recently done a write-up on the in-place upgrading of various distros / BSDs including PC-BSD and OpenBSD.
Lets look first at the PC-BSD attempt, which was done going from 9.2 -> 10.
“I soon found trying to upgrade either the base system or pkg would fail. The update manager did not provide details as to what had gone wrong and so I decided to attempt a manual upgrade by following the FreeBSD Handbook as I had when performing a live upgrade of FreeBSD back in May. At first the manual process seemed to work, downloading the necessary patches for FreeBSD 10 and getting me to resolve conflicts between my existing configuration files and the new versions. Part way through, we are asked to reboot and then continue the upgrade process using the freebsd-update command utility. PC-BSD failed to reboot and, in fact, the boot loader no longer found any operating systems to run.”
Ouch! I’m not sure on the particular commands used, but to lose the boot-loader indicates something went horribly wrong. There is good news in this though. After the pain experienced in the 9.X upgrade process, 11.0 has been vastly improved to help fix this going forward. The updater is also self-updating, which means future changes to tools such as package can be accounted for in previously released versions.
Moving on to OpenBSD, Jesse had much better luck:
> “The documentation provided explains how to upgrade OpenBSD 5.8 to version 5.9 step-by-step and the instructions worked exactly as laid out. Upgrading requires two reboots, one to initiate the upgrade process and one to boot into the new version of OpenBSD. Upgrading the base operating system took approximately ten minutes, including the two reboots. Upgrading the third-party packages took another minute or two. The only quirk I ran into was that I had to manually update my repository mirror information to gain access to the new packages available for OpenBSD 5.9. If this step is not done, then the pkg_add package manager will continue to pull in packages from the old repository we set up for OpenBSD 5.8. “
A good read, and they covered some Linux distros such as Mint and OpenMandriva as well, if you want to find out how they fared.
***
A curated list of awesome DTrace books, articles, videos, tools and resources (http://awesome-dtrace.com/)
The website awesome-dtrace.com compiles a list of resources, including books, articles, videos, tools, and other resources, to help you get the most out of DTrace
The list of books includes 2 open source books that are available on the web, and of course Brendan Gregg’s official DTrace book
There are also cheat sheets, one-liner collections, and a set of DTrace war stories
A breakdown of different PID providers and the userspace statically defined tracepoints
The videos from DTrace.conf 2008, 2012, and soon 2016
And links to the tools to start using DTrace with your favourite programming language, including Erlang, Node.JS, Perl, PHP, Python, or Ruby
There are also DTrace setups for MySQL/MariaDB, and PostreSQL
Joyent has even written a mod_usdt DTrace module for the Apache web server
This seems like a really good resource, and with the efforts of the new OpenDTrace project, to modernize the dtracetoolkit and make it more useful across the different supported operating systems, there has never been a better time to start learning DTrace
***
Installing OpenBSD using a serial console with no external monitor (http://unix.stackexchange.com/questions/292891/how-can-i-install-openbsd-using-the-serial-console-without-external-monitor-wi)
Have you found yourself needing to install OpenBSD from USB, but with a twist, as in no external monitor? Well somebody has and asked the question on stackexchange.
The answer provided is quite well explained, but in a nut-shell the process involves downloading the USB image and making some tweaks before copying it to the physical media.
Specifically with a couple of well-placed echo’s into boot.conf, the serial-port can be enabled and ready for use:
echo "stty com0 115200" > /mnt/etc/boot.conf
echo "set tty com0" >> /mnt/etc/boot.conf
+ After that, simply boot the box and you are ready to access the serial console and drive the installation as normal! #bsdhacks
GSoC 2016 Reports: Split debug symbols for pkgsrc builds (https://blog.netbsd.org/tnf/entry/gsoc_2016_reports_split_debug)
The NetBSD blog provides a status report on one of the GSoC projects that is nearing its midterm evaluation
The project to split debugging data into separate pkgsrc packages, so that users can install the debugging symbols if they need them to debug a failing application
The report is very detailed, and includes “A quick introduction to ELF and how debug information are stored/stripped off”
It walks through the process of writing a simple example application, compiling it, and dealing with the debug data
It includes a number of very useful diagrams, and a summary of what changes needed to be make to the pkgsrc makefile infrastructure
With this as a recipe, someone should be able to do something quite similar for FreeBSD’s ports tree
***
iXsystems
iXsystems’ TrueNAS Firmware Update Delivers Compelling Performance, Replication, and Graphing Improvements (https://www.ixsystems.com/blog/ixsystems-truenas-firmware-update-delivers-compelling-performance-replication-graphing-improvements/)
***
Interview - Jim Brown - jpb@jimby.name (mailto:jpb@jimby.name)
FreeBSD+BBB Sprinkler System
News Roundup
From the past : A Research Unix Reader (http://www.cs.dartmouth.edu/~doug/reader.pdf)
A paper by by Douglas McIlroy
“Selected pages from the nine research editions of the UNIX® Programmer’s Manual illustrate the development of the system”
“Accompanying commentary recounts some of the needs, events, and individual contributions that shaped this evolution.”
Interesting insight into the evolution of the origin UNIX operating system
***
Evolution of C programming practices – Unix 1973–2015 (http://kristerw.blogspot.com/2016/06/evolution-of-c-programming-practices.html)
From the author of the recent post we covered, “20 years of NetBSD code bloat”, comes a new post
“I found a recent paper that also looks at how the BSD code base has evolved, but from a very different perspective compared to my code-size investigation.”
The paper "The Evolution of C Programming Practices: A Study of the Unix Operating System 1973–2015" investigates coding style, and tests seven hypotheses by looking at metrics (line length, number of volatile in the source code, etc.) in 66 releases of Unix from 1973 to 2014. The hypotheses are:
> + Programming practices reflect technology affordances (e.g. developers may be more liberal with screen space when using high resolution displays)
> + Modularity increases with code size
> + New language features are increasingly used to saturation point
> + Programmers trust the compiler for register allocation
> + Code formatting practices converge to a common standard
> + Software complexity evolution follows self correction feedback mechanisms
> + Code readability increases
and the result is that they seem to be true, as interpreted through the metrics.
> “The data points for the releases have somewhat random dates. One issue is that the paper use each release's mean file date (the average of the files' last modification time) instead of the release date (that is why the graphs stop at November 2010, even though FreeBSD 10 was released in 2014). The idea is that this better reflects the age of the code base, but this has the effect of compressing some of the data points (especially the clustering around 1993-1994), and it makes the spline fitting even more suspect.”
> “One other problem is that the original data used by the researchers seems to have incorrect timestamps. For example, 4.3BSD Net/1 was released in 1989, but is listed as 1993-12-25 in the paper. The same is true for at least the Net/2 release too, which was released in 1991, but the paper list it as 1993-07-02.”
***
[old release pictures]
openbsd 2.1 - 5.9, straight from theo's bookshelf. (https://twitter.com/blakkheim/status/747540167112671232)
Speaking of old releases, our Producer JT picked up this gem at Southeast Linuxfest this year (https://twitter.com/q5sys/status/748003859012984837)
Noah Axon shares a scan of his NetBSD 1.4 disc (https://plus.google.com/+NoahAxon/posts/VsiQhUn3tHb)
Jan van den broek shares a pic of his FreeBSD 2.2.5 set (https://plus.google.com/101232368324501316985/posts/4QsaJE2KxXh)
***
FreeBSD: Just in Time (https://bsdmag.org/just_in_time/)
Another BSDMag goodie this week, we have a small article written by Jonathan Garrido which details their experience switching to FreeBSD for a NTP server.
The article is short, but a good read:
> “A Few years ago we had a time problem. Suddenly our linux NTP server, for a reason that I still do not know, started to fail giving us a lot of issues within all the equipment and services within our network. After a quick and brief meeting with management, I found out that there was not sufficient budget left for a fancy and well-suited appliance. So, with no time (literally) and no money to spend, I decided to give it a try and utilized a homemade open source solution, and the operating system of choice was FreeBSD 10.0.”
“Now, let’s pause for a second. You may be thinking, why in the world is this guy doing this, when he has never installed a BSD machine in his life? The answer is very simple; here, in the Dominican Republic, in the heart of the Caribbean, FreeBSD has a very good reputation when it comes to reliability and security. In fact, there is some collective thought within the sysadmin community that says something like: “If you want to deal only once with a service, install it over FreeBSD.””
Jonathan then goes through some of the steps taken to initial deploy NTP services, but with that out of the way, he has a great summary:
> “Fascinated with the whole experience, we migrate one of our internal dns servers to a second FreeBSD machine and at the moment of this writing we are testing haproxy, an open source load-balancing proxy into a another server with the same OS.
> After all this, no time issues have been reported in the past 2 years, so at least for my environment, FreeBSD came just in time.“
***
Beastie Bits
MiniBSD laptop computer (https://hackaday.io/project/643-minibsd-laptop-computer)
The state of LibreSSL in FreeBSD (https://attilagyorffy.com/2016/07/02/the-state-of-libressl-in-freebsd/)
Justin Sherrill is looking for someone willing to run a Go builder with DragonflyBSD (https://www.dragonflydigest.com/2016/07/01/18372.html)
Tiny Unix tools for Windows (https://tinyapps.org/blog/windows/201606040700_tiny_unix_tools_windows.html)
OpenBSD's doas added to the FreeBSD Ports Tree (http://www.freshports.org/security/doas/)
ubuntuBSD 16.04 to feature a combo of BusyBox and OpenRC, no systemd (http://linux.softpedia.com/blog/ubuntubsd-16-04-will-feature-a-combination-of-busybox-and-openrc-but-no-systemd-505463.shtml)
Syncast Podcast 4 : Curl, libcurl and the future of the web, with Daniel Stenberg (http://podcast.sysca.st/podcast/4-curl-libcurl-future-web-daniel-stenberg/)
Feedback/Questions
Harri - Using beadm / zfssnap (http://pastebin.com/qKeCd63F)
Jonathan - bhyve vs Proxmox (http://pastebin.com/EhXDwbWQ)
Mohammad - Bhyve gfx passthrough (http://pastebin.com/ZCNk4Bga)
Jeremy - Shapshots and more Snapshots (http://pastebin.com/xp7nzEYa)
Ron - Microphone (http://pastebin.com/H2xr53CR)
***
| |||
| 01 Apr 2014 | 31: Edgy BSD Users | 01:09:07 | |
This week we'll be talking to Richard Stallman about the upcoming GPLv4 and how it will protect our software from being stolen. After that, we'll show you how to recover from those pesky ZFS on Linux corruption issues, as well as some tips on how to explain to your boss that all the production boxes were compromised. Your questions and all the latest GNUs, on Linux Now - the place to Lin.. ux.
This episode was brought to you by
Headlines
Preorders for cool BSD stuff (http://www.amazon.com/gp/aw/d/0321968972/)
The 2nd edition of The Design and Implementation of the FreeBSD Operating System is up for preorder
We talked to GNN (http://www.bsdnow.tv/episodes/2014_01_29-journaled_news_updates) briefly about it, but he and Kirk (http://www.bsdnow.tv/episodes/2013-10-02_stacks_of_cache) have apparently finally finished the book
"For many years, The Design and Implementation of the FreeBSD Operating System has been recognized as the most complete, up-to-date, and authoritative technical guide to FreeBSD's internal structure. Now, this definitive guide has been extensively updated to reflect all major FreeBSD improvements between Versions 5 and Versions 11"
OpenBSD 5.5 preorders (https://https.openbsd.org/cgi-bin/order) are also up, so you can buy a CD set now
You can help support the project, and even get the -release of the OS before it's available publicly
5.5 is a huge release with lots of big changes, so now is the right time to purchase one of these - tell Austin we sent you!
***
pkgsrcCon 2014 CFP (http://mail-index.netbsd.org/pkgsrc-users/2014/03/18/msg019424.html)
This year's pkgsrcCon is in London, on June 21st and 22nd
There's a Call For Papers out now, so you can submit your talks
Anything related to pkgsrc is fine, it's pretty informal
Does anyone in the audience know if the talks will be recorded? This con is relatively unknown
***
BSDMag issue for March 2014 (http://bsdmag.org/magazine/1860-deploying-netbsd-on-the-cloud-using-aws-ec2-march-bsd-issue)
The monthly BSD magazine releases its newest issue
Topics this time include: deploying NetBSD using AWS EC2, creating a multi-purpose file server with NetBSD, DragonflyBSD as a backup server, more GIMP lessons, network analysis with wireshark and a general security article
The Linux article trend seems to continue... hmm
***
Non-ECC RAM in FreeNAS (http://blog.brianmoses.net/2014/03/why-i-chose-non-ecc-ram-for-my-freenas.html)
We've gotten a few questions about ECC RAM with ZFS
Here we've got a surprising blog post about why someone did not go with ECC RAM for his NAS build
The article mentions the benefits of ECC and admits it is a better choice in nearly all instances, but unfortunately it's not very widespread in consumer hardware motherboards and it's more expensive
Regular RAM also has "special" issues with ZFS and pool corruption
Long post, so check out the whole thing if you've been considering your memory options and weighing the benefits
***
Interview - Pierre Pronchery - khorben@edgebsd.org (mailto:khorben@edgebsd.org) / @khorben (https://twitter.com/khorben)
EdgeBSD (https://www.youtube.com/watch?v=_D_iaad5rPo) (slides (http://ftp.netbsd.org/pub/NetBSD/misc/khorben/asiabsdcon2014/))
Tutorial
Building an OpenBSD desktop (http://www.bsdnow.tv/tutorials/the-desktop-obsd)
News Roundup
Getting to know your portmgr-lurkers (http://blogs.freebsdish.org/portmgr/2014/03/25/getting-to-know-your-portmgr-lurker-frederic-culot)
This week we get to hear from Frederic Culot, colut@
Originally an OpenBSD user from France, Frederic joined as a ports committer in 2010 and recently joined the portmgr lurkers team
"FreeBSD is also one of my sources of inspiration when it comes to how
organizations behave and innovate, and I find it very interesting to compare FreeBSD with
the for-profit companies I work for"
We get to find out a little bit about him, why he loves FreeBSD and what he does for the project
***
NetBSD on the Playstation 2 (https://blog.netbsd.org/tnf/entry/the_playstation2_port_is_back)
Who doesn't want to run NetBSD on their old PS2?
The PS2 port of NetBSD was sadly removed in 2009, but it has been revived
It's using a slightly unusual MIPS CPU that didn't have much GCC support
Hopefully a bootable kernel will be available soon
***
The FreeBSD Challenge update (http://www.thelinuxcauldron.com/2014/03/24/freebsd-challenge-day-22-30/)
Our friend from the Linux Foundation continues his FreeBSD switching journey
This time he starts off by discovering virtual machines suck at keeping accurate time, and some ports weren't working because of his clock being way off
After polling the IRC for help, he finally learns the difference between ntpdate and ntpd and both of their use cases
Maybe he should've just read our NTP tutorial (http://www.bsdnow.tv/tutorials/ntpd)!
***
PCBSD weekly digest (http://blog.pcbsd.org/2014/03/pc-bsd-weekly-feature-digest-23/)
The mount tray icon got lots of updates and fixes
The faulty distribution server has finally been tracked down and... destroyed
New language localization project is in progress
Many many updates to ports and PBIs, new -STABLE builds
***
Feedback/Questions
Antonio writes in (http://slexy.org/view/s27d69qHJW)
Patrick writes in (http://slexy.org/view/s21FhLCHbB)
Chris writes in (http://slexy.org/view/s20Hisk3Yw)
Ron writes in (http://slexy.org/view/s20rBZyTLC)
Tyler writes in (http://slexy.org/view/s2s4CxE4gd)
***
| |||
| 14 Jun 2017 | 198: BSDNorth or You can’t handle the libtruth | 02:14:06 | |
This episode gives you the full dose of BSDCan 2017 recap as well as a blog post on conference speaking advice.
Headlines
Pre-conference activities: Goat BoF, FreeBSD Foundation Board Meeting, and FreeBSD Journal Editorial Board Meeting
The FreeBSD Foundation has a new President as Justin Gibbs is busy this year with building a house, so George Neville-Neil took up the task to serve as President, with Justin Gibbs as Secretary. Take a look at the updated Board of Directors (https://www.freebsdfoundation.org/about/board-of-directors/).
We also have a new staff member (https://www.freebsdfoundation.org/about/staff/): Scott Lamons joined the Foundation team as senior program manager.
Scott’s work for the Foundation will focus on managing and evangelizing programs for advanced technologies in FreeBSD including preparing project plans, coordinating resources, and facilitating interactions between commercial vendors, the Foundation, and the FreeBSD community.
The Foundation also planned various future activities, visits of upcoming conferences, and finding new ways to support and engage the community.
The Foundation now has interns in the form of co-op students from the University of Waterloo, Canada. This is described further in the May 2017 Development Projects Update (https://www.freebsdfoundation.org/blog/may-2017-development-projects-update/). Both students (Siva and Charlie) were also the conference, helping out at the Foundation table, demonstrating the tinderbox dashboard. Follow the detailed instructions (https://www.freebsdfoundation.org/news-and-events/blog/blog-post/building-a-physical-freebsd-build-status-dashboard/) to build one of your own.
The Foundation put out a call for Project Proposal Solicitation for 2017 (https://www.freebsdfoundation.org/blog/freebsd-foundation-2017-project-proposal-solicitation/). If you think you have a good proposal for work relating to any of the major subsystems or infrastructure for FreeBSD, we’d be happy to review it.
Don’t miss the deadlines for travel grants to some of the upcoming conferences. You can find the necessary forms and deadlines at the Travel Grant page (https://www.freebsdfoundation.org/what-we-do/travel-grants/travel-grants/) on the Foundation website.
Pictures from the Goat BoF can be found on Keltia.net (https://assets.keltia.net/photos/BSDCan-2017/Royal%20Oak/index.html)
Overlapping with the GoatBoF, members of the FreeBSD Journal editorial board met in a conference room in the Novotel to plan the upcoming issues. Topics were found, authors identified, and new content was discussed to appeal to even more readers. Check out the FreeBSD Journal website (https://www.freebsdfoundation.org/journal/) and subscribe if you like to support the Foundation in that way.
FreeBSD Devsummit Day 1 & 2 (https://wiki.freebsd.org/DevSummit/201706)
The first day of the Devsummit began with introductory slides by Gordon Tetlow, who organized the devsummit very well.
Benno Rice of the FreeBSD core team presented the work done on the new Code of Conduct, which will become effective soon. A round of Q&A followed, with positive feedback from the other devsummit attendees supporting the new CoC.
After that, Allan Jude joined to talk about the new FreeBSD Community Proposal (FCP) (https://github.com/freebsd/fcp) process. Modelled after IETF RFCs, Joyent RFDs, and Python PEP, it is a new way for the project to reach consensus on the design or implementation of new features or processes. The FCP repo contains FCP#0 that describes the process, and a template for writing a proposal.
Then, the entire core team (except John Baldwin, who could not make it this year) and core secretary held a core Q&A session, Answering questions, gathering feedback and suggestions.
After the coffee break, we had a presentation about Intel’s QAT integration in FreeBSD.
When the lunch was over, people spread out into working groups about BearSSL, Transport (TCP/IP), and OpenZFS.
OpenZFS working group (https://pbs.twimg.com/media/DBu_IMsWAAId2sN.jpg:large):
Matt Ahrens lead the group, and spent most of the first session providing a status update about what features have been recently committed, are out for review, on the horizon, or in the design phase.
Existing Features
Compressed ARC
Compressed Send/Recv
Recently Upstreamed
A recent commit improved RAID-Z write speeds by declaring writes to padding blocks to be optional, and to always write them if they can be aggregated with the next write. Mostly impacts large record sizes.
ABD (ARC buffer scatter/gather)
Upstreaming In Progress
Native Encryption
Channel Programs
Device Removal (Mirrors and Stripes)
Redacted Send/recv
Native TRIM Support (FreeBSD has its own, but this is better and applies to all ZFS implementations)
Faster (mostly sequential) scrub/resilver
DRAID (A great deal of time was spent explaining how this works, with diagrams on the chalk board)
vdev metadata classes (store metadata on SSDs with data is on HDDs, or similar setups. Could also be modified to do dedup to SSD)
Multi-mount protection (“safe import”, for dual-headed storage shelves)
zpool checkpoint (rollback an entire pool, including zfs rename and zfs destroy)
Further Out
Import improvements
Import with missing top-level vdevs (some blocks unreadable, but might let you get some data)
Improved allocator performance -- vdev spacemap log
ZIL performance
Persistent L2ARC
ZSTD Compression
Day 2
Day two started with the Have/Want/Need session for FreeBSD 12.0. A number of features that various people have or are in the process of building, were discussed with an eye towards upstreaming them. Features we want to have in time for 12.0 (early 2019) were also discussed.
After the break was the Vendor summit, which continued the discussion of how FreeBSD and its vendors can work together to make a better operating system, and better products based on it
After lunch, the group broke up into various working groups: Testing/CI, Containers, Hardening UFS, and GELI Improvements
Allan lead the GELI Improvements session. The main thrust of the discussions was fixing an outstanding bug in GELI when using both key slots with passphrases. To solve this, and make GELI more extensible, the metadata format will be extended to allow it to store more than 512 bytes of data (currently 511 bytes are used).
The new format will allow arbitrarily large metadata, defined at creation time by selecting the number of user key slots desired.
The new extended metadata format will contain mostly the same fields, except the userkey will no longer be a byte array of IV-key, Data-key, HMAC, but a struct that will contain all data about that key
This new format will store the number of pkcs5v2 iterations per key, instead of only having a single location to store this number for all keys (the source of the original bug)
A new set of flags per key, to control some aspects of the key (does it require a keyfile, etc), as well as possibly the role of the key.
An auxdata field related to the flags, this would allow a specific key with a specific flag set, to boot a different partition, rather than decrypt the main partition.
A URI to external key material is also stored per key, allowing GELI to uniquely identify the correct data to load to be able to use a specific decryption key
And the three original parts of the key are stored in separate fields now. The HMAC also has a type field, allowing for a different HMAC algorithm to be used in the future.
The main metadata is also extended to include a field to store the number of user keys, and to provide an overall HMAC of the metadata, so that it can be verified using the master key (provide any of the user keys)
Other topics discussed:
Ken Merry presented sedutil, a tool for managing Self Encrypting Drives, as may be required by certain governments and other specific use cases.
Creating a deniable version of GELI, where the metadata is also encrypted
The work to implemented GELI in the UEFI loader was discussed, and a number of developers volunteered to review and test the code
Following the end of the Dev Summit, the “Newcomers orientation and mentorship” session was run by Michael W. Lucas, which attempts to pair up first time attendees with oldtimers, to make sure they always know a few people they can ask if they have questions, or if they need help getting introduced to the right people.
News Roundup
Conference Day 1 (http://www.bsdcan.org/2017/schedule/day_2017-06-09.en.html)
The conference opened with some short remarks from Dan Langille, and then the opening keynote by Dr Michael Geist, a law professor at the University of Ottawa where he holds the Canada Research Chair in Internet and E-commerce Law. The keynote focused on what some of the currently issues are, and how the technical community needs to get involved at all levels. In Canada especially, contacting your representatives is quite effective, and when it does not happen, they only hear the other side of the story, and often end up spouting talking points from lobbyists as if they were facts.
The question period for the keynote ran well overtime because of the number of good questions the discussion raised, including how do we fight back against large telcos with teams of lawyers and piles of money.
Then the four tracks of talks started up for the day
The day wrapped up with the Work In Progress (WIP) session.
Allan Jude presented work on ZSTD compression in ZFS
Drew Gallatin presented about work at Netflix on larger mbufs, to avoid the need for chaining and to allow more data to be pushed at once. Results in an 8% CPU time reduction when pushing 90 gbps of TLS encrypted traffic
Dan Langille presented about letsencrypt (the acme.sh tool specifically), and bacula
Samy Al Bahra presented about Concurrency Kit
***
Conference Day 2 (http://www.bsdcan.org/2017/schedule/day_2017-06-10.en.html)
Because Dan is a merciful soul, BSDCan starts an hour later on the second day
Another great round of talks and BoF sessions over lunch
The hallway track was great as always, and I spent most of the afternoon just talking with people
Then the final set of talks started, and I was torn between all four of them
Then there was the auction, and the closing party
***
BSDCan 2017 Auction Swag (https://blather.michaelwlucas.com/archives/2962)
Groff Fundraiser Pins: During the conference, You could get a unique Groff pin, by donating more than the last person to either the FreeBSD or OpenBSD foundation
Michael W. Lucas and his wife Liz donated some interesting home made and local items to the infamous Charity Auction
I donated the last remaining copy of the “Canadian Edition” of “FreeBSD Mastery: Advanced ZedFS”, and a Pentium G4400 (Skylake) CPU (Supports ECC or non-ECC)
Peter Hessler donated his pen (Have you read “Git Commit Murder” yet?)
Theo De Raadt donated his autographed conference badge
David Maxwell donated a large print of the group photo from last years FreeBSD Developers Summit, which was purchased by Allan
There was also a FreeBSD Dev Summit T-Shirt (with the Slogan: What is Core doing about it?) autographed by all of the attending members of core, with a forged jhb@ signature.
Lastly, someone wrote “I <3 FreeBSD” on a left over conference t-shirt with magic marker, and the bidding began to make OpenBSD developer Henning Brauer wear it to the closing party. The top bid was $150 by Kristof Provost, the FreeBSD pf maintainer.
***
Henning Brauer loves FreeBSD (https://twitter.com/henningBrauer)
In addition to the $150 donation that resulted in Henning wearing the I love FreeBSD t-shirt, he also took selfies with people in exchange for an additional donation of $10. A total of over $500 was raised.
Michael W. Lucas (https://twitter.com/mwlauthor/status/874656462433386497)
Michael Dexter (https://twitter.com/michaeldexter/status/874344686885904384)
FreeBSD Foundation Interns + Ed Maste + Eric Joyner (https://twitter.com/yzgyyang/status/873714734343880705)
Pierre Ponchery (https://twitter.com/khorben/status/873673295903825925)
Nick Danger (https://twitter.com/niqdanger/status/873697176513380353)
Michael Shirk (https://twitter.com/shirkdog/status/873687910175866881)
Calvin Hendryx-Parker (https://twitter.com/calvinhp/status/873686591692255233)
Reyk Floeter (https://twitter.com/reykfloeter/status/873673717884346368)
Rod Grimes (https://twitter.com/akpoff/status/873673432751370240)
Jim Thompson (https://twitter.com/gonzopancho/status/873700951651233792)
Sean Chittenden and Sam Gwydir, Henning wearing Theo de Raadt’s badge (https://twitter.com/SeanChittenden/status/873750297113501697)
David Duncan (https://twitter.com/davdunc/status/873807305162334208)
***
libtrue (https://github.com/libtrue/libtrue)
At the hacker lounge, a joke email was sent to the FreeBSD developers list, making it look like a change to true(1)’s manpage had been committed to “document that true(1) supports libxo”
While the change was not actually made, as you might expect this started a discussion about if this was really necessary.
This spawned a new github repo
While this all started as a joke, it then became an example of how rapid collaboration can happen, and an example of implementing a number of modern technologies in FreeBSD, including libxo (json output), and capsicum (security sandboxing)
The project has an large number of open issues and enhancement suggestions, and a number of active pull requests including:
Add Vagrantfile and Ansible playbooks for VM
DTrace Support (Add the trueprov provider to allow tracing of true.
A Code of Conduct
libtrue.xyz website as a git submodule
a false binary
Python and Go bindings
***
On Conference Speaking (https://hynek.me/articles/speaking/)
Phase 1: Idea
Until now I’ve never had to sit down and ponder what I could speak about. Over the year, I run into at least one topic I know something about that appears to be interesting to the wider public. I’m positive that’s true for almost anyone if they keep their minds open and keep looking for it.
Phase 2: Call for Proposals
In the end I have to come up with a good pitch that speaks to as many people as possible and with a speculative outline. Since there are always many more submissions than talk slots, this is the first critical point. There are many reasons why a proposal can be refused, so put effort into not giving the program committee any additional, that are entirely avoidable.
Phase 3: Waiting for the CFP Result
...do passive research: if I see something relevant, I add it to my mind map. At this point my mind map looks atypical though: it has a lot of unordered root nodes. I just throw in everything that looks interesting and add some of my own thoughts to it. In the case of the reliability topic, I spend a lot of time to stay on top it anyway so a lot of material emerged.
Phase 4: The Road to an Outline
Once the talk is accepted, research intensifies. Books and articles I’ve written down for further research are read and topics extracted. In 2017, this started on January 23. But before I start writing, I get the mind map into a shape that makes sense and that will support me. This is the second critical point: I have to come up with a compelling story using the material I’ve collected. Just enumerating facts and wisdom a good talk doesn’t make. It has to have a good flow that makes sense and that keeps people engaged.
Phase 5: Slides
I have a very strong opinion on slides: use few, big words. Don’t make people read your slides unless it’s code samples. Otherwise they’ll be distracted from what you say. You’ll see me rarely use fonts smaller than 100pt (code ~40–60pt) which is readable from everywhere and forces me to be as brief as possible.
Phase 6: Polishing
I firmly believe that this phase makes or breaks a talk. Only by practicing again and again you’ll notice rough spots, weak transitions, and redundancies. Each iteration makes the talk a bit better both regarding the slides and my ability to present it. Each iteration adds impressions that my subconscious mind chews on and makes things fall in place and give me inspiration in unlikely moments.
Phase 7: Sneak Preview
In the past years I was blessed with the opportunity to test my talks in front of a smaller audiences. Interestingly, I’ve come to take smaller events more seriously than the big ones. If a small conference pays for my travels and gives me a prominent slot, I have both more responsibility and attention than if I paid my way to an event where I’m one of many speakers.
Phase 8: Refinement and More Polishing
The first session is always just going through all slides, reacquainting myself with my deck. Then it always takes quite a bit of willpower until I do a full practice run again: the first time always pretty brutal because I tend to forget pretty fast. On the other hand, it rather quickly comes back too. Which makes it even harder to motivate myself to start. Ideally I’d have access to a video recording from phase 7 to have a closer look at what work could be improved. But since it’s usually smaller conferences, I don’t.
Phase 9: Travel
Whenever I travel to conferences I bring everything I need for my talk and then some. To make sure I don’t forget anything essential, I have a packing list for my business trips (and vacations too – the differences are so minimal that I use a unified list). My epic packing list for business trips. I print it out the day before departure and cross stuff off as I pack. I highly recommend to anyone to emulate this since packing is a lot less stressful if you just follow a checklist.
Phase 10: Showtime!
So this is it. The moment everything else led to. People who suffer from fear of public speaking think this is the worst part. But if you scroll back you’ll realize: this is the payoff! This is what you worked toward. This is the fun, easy part. Once you stand in front of the audience, the work is done and you get to enjoy the ride.
Beastie Bits
Kris and Ken Moore - TrueOS Q&A (https://www.trueos.org/blog/discourse-trueos-qa-61617/)
New home for the repository conversions (NetBSD Git mirror is now on GitHub.com/NetBSD) (https://mail-index.netbsd.org/tech-repository/2017/06/10/msg000637.html)
Tab completion in OpenBSD's ksh (https://deftly.net/posts/2017-05-01-openbsd-ksh-tab-complete.html)
pkgsrcCon July 1&2 (http://pkgsrc.org/pkgsrcCon/2017/)
OpenBSD 6.1 syspatch installed SP kernel on MP system (https://www.mail-archive.com/misc@openbsd.org/msg153421.html)
KNOXBug meeting this Friday (http://knoxbug.org/content/join-us-freebsd-day)
***
Feedback/Questions
Rob - FreeNAS Corral (http://dpaste.com/2XEE9JA#wrap)
Brad - ZFS snapshot strategy (http://dpaste.com/27GSJK0#wrap)
Phil - ZFS Send via Snail Mail (http://dpaste.com/3D02RYZ#wrap)
Phillip - Network Limits for Public NTP Server (http://dpaste.com/0ZSMVWH#wrap)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 21 May 2014 | 38: A BUG's Life | 01:28:34 | |
We're back from BSDCan! This week on the show we'll be chatting with Brian Callahan and Aaron Bieber about forming a local BSD users group. We'll get to hear their experiences of running one and maybe encourage some of you to start your own! After that, we've got a tutorial on the basics of NetBSD's package manager, pkgsrc. Answers to your emails and the latest headlines, on BSD Now - the place to B.. SD.
This episode was brought to you by
Headlines
FreeBSD 11 goals and discussion (http://blather.michaelwlucas.com/archives/2053)
Something that actually happened at BSDCan this year...
During the FreeBSD devsummit, there was some discussion about what changes will be made in 11.0-RELEASE
Some of MWL's notes include: the test suite will be merged to 10-STABLE, more work on the MIPS platforms, LLDB getting more attention, UEFI boot and install support
A large list of possibilities was also included and open for discussion, including AES-GCM in IPSEC, ASLR, OpenMP, ICC, in-place kernel upgrades, Capsicum improvements, TCP performance improvements and A LOT more
There's also some notes from the devsummit virtualization session (http://blather.michaelwlucas.com/archives/2060), mostly talking about bhyve
Lastly, he also provides some notes about ports and packages (http://blather.michaelwlucas.com/archives/2065) and where they're going
***
An SSH honeypot with OpenBSD and Kippo (http://securit.se/2014/05/how-to-install-kippo-ssh-honeypot-on-openbsd-5-5-with-chroot/)
Everyone loves messing with script kiddies, right?
This blog post introduces Kippo (https://code.google.com/p/kippo/), an SSH honeypot tool, and how to use it in combination with OpenBSD
It includes a step by step (or rather, command by command) guide and some tips for running a honeypot securely
You can use this to get new 0day exploits or find weaknesses in your systems
OpenBSD makes a great companion for security testing tools like this with all its exploit mitigation techniques that protect all running applications
***
NetBSD foundation financial report (https://www.netbsd.org/foundation/reports/financial/2013.html)
The NetBSD foundation has posted their 2013 financial report
It's a very "no nonsense" page, pretty much only the hard numbers
In 2013, they got $26,000 of income in donations
The rest of the page shows all the details, how they spent it on hardware, consulting, conference fees, legal costs and everything else
Be sure to donate to whichever BSDs you like and use!
***
Building a fully-encrypted NAS with OpenBSD (http://www.geektechnique.org/projectlab/796/how-to-build-a-fully-encrypted-nas-on-openbsd.html)
Usually the popular choice for a NAS system is FreeNAS, or plain FreeBSD if you know what you're doing
This article takes a look at the OpenBSD side and explains how (http://www.geektechnique.org/projectlab/797/openbsd-encrypted-nas-howto.html) to build a NAS with security in mind
The NAS will be fully encrypted, no separate /boot partition like FreeBSD and FreeNAS require - this means the kernel itself is even protected
The obvious trade-off is the lack of ZFS support for storage, but this is an interesting idea that would fit most people's needs too
There's also a bit of background information on NAS systems in general, some NAS-specific security tips and even some nice graphs and pictures of the hardware - fantastic write up!
***
Interview - Brian Callahan & Aaron Bieber - admin@lists.nycbug.org (mailto:admin@lists.nycbug.org) & admin@cobug.org (mailto:admin@cobug.org)
Forming a local BSD Users Group
Tutorial
The basics of pkgsrc (http://www.bsdnow.tv/tutorials/pkgsrc)
News Roundup
FreeBSD periodic mails vs. monitoring (http://deranfangvomende.wordpress.com/2014/05/11/freebsd-periodic-mails-vs-monitoring/)
If you've ever been an admin for a lot of FreeBSD boxes, you've probably noticed that you get a lot of email
This page tells about all the different alert emails, cron emails and other reports you might end up getting, as well as how to manage them
From bad SSH logins to Zabbix alerts, it all adds up quickly
It highlights the periodic.conf file and FreeBSD's periodic daemon, as well as some third party monitoring tools you can use to keep track of your servers
***
Doing cool stuff with OpenBSD routing domains (http://www.skogsrud.net/?p=44)
A blog post from our viewer and regular emailer, Kjell-Aleksander!
He manages some internally-routed IP ranges at his work, but didn't want to have equipment for each separate project
This is where OpenBSD routing domains and pf come in to save the day
The blog post goes through the process with all the network details you could ever dream of
He even named his networking equipment... after us (http://i.imgur.com/penYQFP.jpg)
***
LibreSSL, the good and the bad (http://insanecoding.blogspot.com/2014/04/libressl-good-and-bad.html)
We're all probably familiar with OpenBSD's fork of OpenSSL at this point
However, "for those of you that don't know it, OpenSSL is at the same time the best and most popular SSL/TLS library available, and utter junk"
This article talks about some of the cryptographic development challenges involved with maintaining such a massive project
You need cryptographers, software engineers, software optimization specialists - there are a lot of roles that need to be filled
It also mentions some OpenSSL alternatives and recent LibreSSL progress, as well as some downsides to the fork - the main one being their aim for backwards compatibility
***
PCBSD weekly digest (http://blog.pcbsd.org/2014/05/weekly-feature-digest-28-photos-of-the-new-appcafe-re-design/)
Lots going on in PCBSD land this week, AppCafe has been redesigned
The PBI system is being replaced with pkgng, PBIs will be automatically converted once you update
In the more recent post (http://blog.pcbsd.org/2014/05/weekly-feature-digest-29-pbing/), there's some further explanation of the PBI system and the reason for the transition
It's got lots of details on the different ways to install software, so hopefully it will clear up any possible confusion
***
Feedback/Questions
Antonio writes in (http://slexy.org/view/s2UbEhgjce)
Daniel writes in (http://slexy.org/view/s21XU0y3JP)
Sean writes in (http://slexy.org/view/s2QQtuawFl)
tsyn writes in (http://slexy.org/view/s20XrT5Q8U)
Chris writes in (http://slexy.org/view/s2ayZ1nsdv)
***
| |||
| 22 Feb 2017 | 182: Bloaty McBloatface | 01:06:58 | |
This week on the show, we’ve got FreeBSD quarterly Status reports to discuss, OpenBSD changes to the installer, EC2 and IPv6 and more. Stay
This episode was brought to you by
Headlines
OpenBSD changes of note 6 (http://www.tedunangst.com/flak/post/openbsd-changes-of-note-6)
OpenBSD can now be cross built with clang. Work on this continues
Build ld.so with -fno-builtin because otherwise clang would optimize the local versions of functions like dlmemset into a call to memset, which doesn’t exist.
Add connection timeout for ftp (http). Mostly for the installer so it can error out and try something else.
Complete https support for the installer.
I wonder how they handle certificate verification. I need to look into this as I’d like to switch the FreeBSD installer to this as well
New ocspcheck utility to validate a certificate against its ocsp responder.
net lock here, net lock there, net lock not quite everywhere but more than before.
More per cpu counters in networking code as well.
Disable and lock Silicon Debug feature on modern Intel CPUs.
Prevent wireless frame injection attack described at 33C3 in the talk titled “Predicting and Abusing WPA2/802.11 Group Keys” by Mathy Vanhoef.
Add support for multiple transmit ifqueues per network interface. Supported drivers include bge, bnx, em, myx, ix, hvn, xnf.
pledge now tracks when a file as opened and uses this to permit or deny ioctl.
Reimplement httpd’s support for byte ranges. Fixes a memory DOS.
FreeBSD 2016Q4 Status Report (https://www.freebsd.org/news/status/report-2016-10-2016-12.html)
An overview of some of the work that happened in October - December 2016
The ports tree saw many updates and surpassed 27,000 ports
The core team was busy as usual, and the foundation attended and/or sponsored a record 24 events in 2016.
CEPH on FreeBSD seems to be coming along nicely. For those that do not know, CEPH is a distributed filesystem that can sit on top of another filesystem. That is, you can use it to create a clustered filesystem out of a bunch of ZFS servers. Would love to have some viewers give it a try and report back.
OpenBSM, the FreeBSD audit framework, got some updates
Ed Schouten committed a front end to export sysctl data in a format usable by Prometheus, the open source monitoring system. This is useful for other monitoring software too.
Lots of updates for various ARM boards
There is an update on Reproducible Builds in FreeBSD, “ It is now possible to build the FreeBSD base system (kernel and userland) completely reproducibly, although it currently requires a few non-default settings”, and the ports tree is at 80% reproducible
Lots of toolchain updates (gcc, lld, gdb)
Various updates from major ports teams
***
Amazon rolls out IPv6 support on EC2 (http://www.daemonology.net/blog/2017-01-26-IPv6-on-FreeBSD-EC2.html)
A few hours ago Amazon announced that they had rolled out IPv6 support in EC2 to 15 regions — everywhere except the Beijing region, apparently. This seems as good a time as any to write about using IPv6 in EC2 on FreeBSD instances.
First, the good news: Future FreeBSD releases will support IPv6 "out of the box" on EC2. I committed changes to HEAD last week, and merged them to the stable/11 branch moments ago, to have FreeBSD automatically use whatever IPv6 addresses EC2 makes available to it.
Next, the annoying news: To get IPv6 support in EC2 from existing FreeBSD releases (10.3, 11.0) you'll need to run a few simple commands. I consider this unfortunate but inevitable: While Amazon has been unusually helpful recently, there's nothing they could have done to get support for their IPv6 networking configuration into FreeBSD a year before they launched it.
You need the dual-dhclient port:
pkg install dual-dhclient
And the following lines in your /etc/rc.conf:
ifconfigDEFAULT="SYNCDHCP acceptrtadv"
ipv6activateallinterfaces="YES"
dhclientprogram="/usr/local/sbin/dual-dhclient"
+ It is good to see FreeBSD being ready to use this feature on day 0, not something we would have had in the past
Finally, one important caveat: While EC2 is clearly the most important place to have IPv6 support, and one which many of us have been waiting a long time to get, this is not the only service where IPv6 support is important. Of particular concern to me, Application Load Balancer support for IPv6 is still missing in many regions, and Elastic Load Balancers in VPC don't support IPv6 at all — which matters to those of us who run non-HTTP services. Make sure that IPv6 support has been rolled out for all the services you need before you start migrating.
Colin’s blog also has the details on how to actually activate IPv6 from the Amazon side, if only it was as easy as configuring it on the FreeBSD side
***
FreeBSD’s George Neville-Neil tries valiantly for over an hour to convince a Linux fan of the error of their ways (https://www.youtube.com/watch?v=cofKxtIO3Is)
In today's episode of the Lunduke Hour I talk to George Neville-Neil -- author and FreeBSD advocate. He tries to convince me, a Linux user, that FreeBSD is better.
+ They cover quite a few topics, including:
+ licensing, and the motivations behind it
+ vendor relations
+ community
+ development model
+ drivers and hardware support
+ George also talks about his work with the FreeBSD Foundation, and the book he co-authored, “The Design and Implementation of the FreeBSD Operating System, 2nd Edition”
News Roundup
An interactive script that makes it easy to install 50+ desktop environments following a base install of FreeBSD 11 (https://github.com/rosedovell/unixdesktops)
And I thought I was doing good when I wrote a patch for the installer that enables your choice of 3 desktop environments...
This is a collection of scripts meant to install desktop environments on unix-like operating systems following a base install. I call one of these 'complete' when it meets the following requirements:
+ A graphical logon manager is presented without user intervention after powering on the machine
+ Logging into that graphical logon manager takes the user into the specified desktop environment
+ The user can open a terminal emulator
I need to revive my patch, and add Lumina to it
***
Firefox 51 on sparc64 - we did not hit the wall yet (https://blog.netbsd.org/tnf/entry/firefox_51_on_sparc64_we)
A NetBSD developers tells the story of getting Firefox 51 running on their sparc64 machine
It turns out the bug impacted amd64 as well, so it was quickly fixed
They are a bit less hopeful about the future, since Firefox will soon require rust to compile, and rust is not working on sparc64 yet
Although there has been some activity on the rust on sparc64 front, so maybe there is hope
The post also look at a few alternative browsers, but it not hopeful
***
Introducing Bloaty McBloatface: a size profiler for binaries (http://blog.reverberate.org/2016/11/07/introducing-bloaty-mcbloatface.html)
I’m very excited to announce that today I’m open-sourcing a tool I’ve been working on for several months at Google. It’s called Bloaty McBloatface, and it lets you explore what’s taking up space in your .o, .a, .so, and executable binary files.
Bloaty is available under the Apache 2 license. All of the code is available on GitHub: github.com/google/bloaty. It is quick and easy to build, though it does require a somewhat recent compiler since it uses C++11 extensively. Bloaty primarily supports ELF files (Linux, BSD, etc) but there is some support for Mach-O files on OS X too. I’m interested in expanding Bloaty’s capabilities to more platforms if there is interest!
I need to try this one some of the boot code files, to see if there are places we can trim some fat
We’ve been using Bloaty a lot on the Protocol Buffers team at Google to evaluate the binary size impacts of our changes. If a change causes a size increase, where did it come from? What sections/symbols grew, and why? Bloaty has a diff mode for understanding changes in binary size
The diff mode looks especially interesting. It might be worth setting up some kind of CI testing that alerts if a change results in a significant size increase in a binary or library
***
A BSD licensed mdns responder (https://github.com/kristapsdz/mdnsd)
One of the things we just have to deal with in the modern world is service and system discovery. Many of us have fiddled with avahi or mdnsd and related “mdns” services.
For various reasons those often haven’t been the best-fit on BSD systems.
Today we have a github project to point you at, which while a bit older, has recently been updated with pledge() support for OpenBSD.
First of all, why do we need an alternative? They list their reasons:
This is an attempt to bring native mdns/dns-sd to OpenBSD. Mainly cause all the other options suck and proper network browsing is a nice feature these days.
Why not Apple's mdnsd ?
1 - It sucks big time.
2 - No BSD License (Apache-2).
3 - Overcomplex API.
4 - Not OpenBSD-like.
Why not Avahi ?
1 - No BSD License (LGPL).
2 - Overcomplex API.
3 - Not OpenBSD-like
4 - DBUS and lots of dependencies.
Those already sound like pretty compelling reasons. What makes this “new” information again is the pledge support, and perhaps it’s time for more BSD’s to start considering importing something like mdnsd into their base system to make system discovery more “automatic”
***
Beastie Bits
Benno Rice at Linux.Conf.Au: The Trouble with FreeBSD (https://www.youtube.com/watch?v=Ib7tFvw34DM)
State of the Port of VMS to x86 (http://vmssoftware.com/pdfs/State_of_Port_20170105.pdf)
Microsoft Azure now offers Patent Troll Protection (https://thestack.com/cloud/2017/02/08/microsoft-azure-now-offers-patent-troll-ip-protection/)
FreeBSD Storage Summit 2017 (https://www.freebsdfoundation.org/news-and-events/event-calendar/freebsd-storage-summit-2017/)
If you are going to be in Tokyo, make sure you come to (http://bhyvecon.org/)
Feedback/Questions
Farhan - Laptops (http://pastebin.com/bVqsvM3r)
Hjalti - rclone (http://pastebin.com/7KWYX2Mg)
Ivan - Jails (http://pastebin.com/U5XyzMDR)
Jungle - Traffic Control (http://pastebin.com/sK7uEDpn)
***
| |||
| 13 Mar 2025 | 602: Wildcard Gotchas | 00:57:11 | |
I Tried FreeBSD as a Desktop in 2025. Here's How It Went, Cray 1 Supercomputer Performance Comparisons With Home Computers Phones and Tablets, The first perfect computer, Find Name Wildcard Gotcha, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
I Tried FreeBSD as a Desktop in 2025. Here's How It Went (https://www.howtogeek.com/i-tried-freebsd-as-a-desktop-heres-how-it-went/)
Cray 1 Supercomputer Performance Comparisons With Home Computers Phones and Tablets (http://www.roylongbottom.org.uk/Cray%201%20Supercomputer%20Performance%20Comparisons%20With%20Home%20Computers%20Phones%20and%20Tablets.htm)
News Roundup
State of virtualizing the BSDs on Apple Silicon (https://briancallahan.net/blog/20250222.html)
The first perfect computer (https://celso.io/posts/2025/01/26/the-first-perfect-computer/)
Find Name Wildcard Gotcha (https://utcc.utoronto.ca/~cks/space/blog/unix/FindNameWildcardGotcha)
New Patreon Levels
Level 1 - user memory (Tip Jar) @ $1 / month
Show your support for the show
Level 2 - virtual memory (Ad-Free Episodes) @ $5 / month
Ad-free episodes
Level 3 - kmem (VIP Patron) @ $10 / month
Everything in higher memory levels &
Your feedback and questions jump the queue and go in the next episode.
Personal shout outs (with your consent) for recommending articles we cover.
Level 4 - physical memory @ $20 / month
What's included:
Everything in higher memory levels &
You can send in audio/video questions and we'll air your audio in the show feedback section (if the quality of your recording is decent)
Behind-the-scenes content - Raw Video from Recording sessions with intro/outro discussion not included in the show
Additional Content when we all make it
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 10 Sep 2020 | 367: Changing jail datasets | 00:45:28 | |
A 35 Year Old Bug in Patch, Sandbox for FreeBSD, Changing from one dataset to another within a jail, You don’t need tmux or screen for ZFS, HardenedBSD August 2020 Status Report and Call for Donations, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/)
Headlines
A 35 Year Old Bug in Patch (http://bsdimp.blogspot.com/2020/08/a-35-year-old-bug-in-patch-found-in.html)
Larry Wall posted patch 1.3 to mod.sources on May 8, 1985. A number of versions followed over the years. It's been a faithful alley for a long, long time. I've never had a problem with patch until I embarked on the 2.11BSD restoration project. In going over the logs very carefully, I've discovered a bug that bites this effort twice. It's quite interesting to use 27 year old patches to find this bug while restoring a 29 year old OS...
Sandbox for FreeBSD (https://www.relkom.sk/en/fbsd_sandbox.shtml)
A sandbox is a software which artificially limits access to the specific resources on the target according to the assigned policy. The sandbox installs hooks to the kernel syscalls and other sub-systems in order to interrupt the events triggered by the application. From the application point of view, application working as usual, but when it wants to access, for instance, /dev/kmem the sandbox software decides against the assigned sandbox scheme whether to grant or deny access.
In our case, the sandbox is a kernel module which uses MAC (Mandatory Access Control) Framework developed by the TrustedBSD team. All necessary hooks were introduced to the FreeBSD kernel.
Source Code (https://gitlab.com/relkom/sandbox)
Documentation (https://www.relkom.sk/en/fbsd_sandbox_docs.shtml)
News Roundup
Changing from one dataset to another within a jail (https://dan.langille.org/2020/08/16/changing-from-one-dataset-to-another-within-a-freebsd-iocage-jail/)
ZFS has a the ability to share itself within a jail. That gives the jail some autonomy, and I like that.
I’ve written briefly about that, specifically for iocage. More recently, I started using a zfs snapshot for caching clearing.
The purpose of this post is to document the existing configuration of the production FreshPorts webserver and outline the plan on how to modify it for more zfs-snapshot-based cache clearing.
You don’t need tmux or screen for ZFS (https://rubenerd.com/you-dont-need-tmux-or-screen-for-zfs/)
Back in January I mentioned how to add redundancy to a ZFS pool by adding a mirrored drive. Someone with a private account on Twitter asked me why FreeBSD—and NetBSD!—doesn’t ship with a tmux or screen equivilent in base in order to daemonise the process and let them run in the background.
ZFS already does this for its internal commands.
HardenedBSD August 2020 Status Report and Call for Donations (https://hardenedbsd.org/article/shawn-webb/2020-08-15/hardenedbsd-august-2020-status-report-and-call-donations)
This last month has largely been a quiet one. I've restarted work on porting five-year-old work from the Code Pointer Integrity (CPI) project into HardenedBSD. Chiefly, I've started forward-porting the libc and rtld bits from the CPI project and now need to look at llvm compiler/linker enhancements. We need to be able to apply SafeStack to shared objects, not just application binaries. This forward-porting work I'm doing is to support that effort.
The infrastructure has settled and is now churning normally and happily. We're still working out bandwidth issues. We hope to have a new fiber line ran by the end of September.
As part of this status report, I'm issuing a formal call for donations. I'm aiming for $4,000.00 USD for a newer self-hosted Gitea server. I hope to purchase the new server before the end of 2020.
Important parts of Unix's history happened before readline support was common (https://utcc.utoronto.ca/~cks/space/blog/unix/TimeBeforeReadline)
Unix and things that run on Unix have been around for a long time now. In particular, GNU Readline was first released in 1989 (as was Bash), which is long enough ago for it (or lookalikes) to become pretty much pervasive, especially in Unix shells. Today it's easy to think of readline support as something that's always been there. But of course this isn't the case. Unix in its modern form dates from V7 in 1979 and 4.2 BSD in 1983, so a lot of Unix was developed before readline and was to some degree shaped by the lack of it.
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Mason - mailserver (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/367/feedback/Mason%20-%20mailserver.md)
casey - freebsd on decline (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/367/feedback/casey%20-%20freebsd%20on%20decline.md)
denis - postgres (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/367/feedback/denis%20-%20postgres.md)
***
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 18 Oct 2017 | 216: Software is storytelling | 01:49:21 | |
EuroBSDcon trip report, how to secure OpenBSD’s LDAP server, ZFS channel programs in FreeBSD HEAD and why software is storytelling.
This episode was brought to you by
Headlines
EuroBSDcon Trip Report
This is from Frank Moore, who has been supplying us with collections of links for the show and who we met at EuroBSDcon in Paris for the first time. Here is his trip report.
My attendance at the EuroBSDCon 2017 conference in Paris was sprinkled with
several 'firsts'. My first visit to Paris, my first time travelling on a EuroTunnel Shuttle train and my first time at any BSD conference. Hopefully, none of these will turn out to be 'lasts'.
I arrived on the Wednesday afternoon before the conference started on Thursday morning. My hotel was conveniently located close to the conference centre in Paris' 3rd arrondissement. This area is well-known as a buzzy enclave of hip cafes, eateries, independent shops, markets, modern galleries and museums. It certainly lived up to its reputation. Even better, the weather held over the course of the conference, only raining once, with the rest of the time being both warm and sunny.
The first two days were taken up with attending Dr Kirk McKusick's excellent tutorial 'An Introduction to the FreeBSD Open-Source Operating System'. This is training "straight from the horse's mouth". Kirk has worked extensively on The FreeBSD operating system since the 1980's, helping to design the original BSD filesystem (FFS) and later working on UFS as well. Not only is Kirk an engaging speaker, making what could be a dry topic very interesting, he also
sprinkles liberal doses of history and war stories throughout his lectures. Want to know why a protocol was designed the way that it was? Or why a system flag has a particular value or position in a record? Kirk was there and has the first-hand answer. He reminisces about his meetings and work with other Unix and BSD luminaries and debunks and confirms common myths in equal measure.
Kirk's teaching style and knowledge are impressive. Every section starts with an overview and a big picture diagram before drilling down into the nitty-gritty detail. Nothing feels superfluous, and everything fits together logically. It's easy to tell that the material and its delivery have been honed over many years, but without feeling stale. Topics covered included the kernel, processes, virtual memory, threads, I/O, devices, FFS, ZFS, and networking.
The slides were just as impressive, with additional notes written by a previous student and every slide containing a reference back to the relevant page(s) in the 2nd edition of Kirk's operating system book. As well as a hard copy for those that requested it, Kirk also helpfully supplied soft copies of all the training materials.
The breaks in between lectures were useful for meeting the students from the other tutorials and for recovering from the inevitable information overload.
It's not often that you can get to hear someone as renowned as Dr McKusick give a lecture on something as important as the FreeBSD operating system. If you have any interest in FreeBSD, Unix history, or operating systems in general, I would urge you to grab the opportunity to attend one of his lectures. You won't be disappointed.
The last two days of the conference consisted of various hour-long talks by members of each of the main BSD systems. All of them were fairly evenly represented except Dragonfly BSD which unfortunately only had one talk. With three talks going on at any one time, it was often difficult to pick which one to go to. At other times there might be nothing to pique the interest. Attendance at a talk is not mandatory, so for those times when no talks looked inviting, just hanging out in one of the lobby areas with other attendees was often just as interesting and informative.
The conference centre itself was certainly memorable with the interior design of an Egyptian temple or pyramid. All the classrooms were more than adequate while the main auditorium was first-class and easily held the 300+ attendees comfortably. All in all, the facilities, catering and organisation were excellent. Kudos to the EuroBSDCon team, especially Bapt and Antoine for all their hard work and hospitality.
As a long-time watcher and occasional contributor to the BSD Now podcast it was good to meet both Allan and Benedict in the flesh. And having done some proofreading for Michael Lucas previously, it was nice to finally meet him as well.
My one suggestion to the organisers of the next conference would be to provide more
hand-holding for newbies. As a first-time attendee at a BSD conference it would have been nice to have been formally introduced to various people within the projects as the goto people for their areas. I could do this myself, but it's not always easy finding the right person and wrangling an introduction. I also think it was a missed opportunity for each project to recruit new developers to their cause. Apparently, this is already in place at BSDCan, but should probably be rolled out across all BSD conferences.
Having said all that, my aims for the conference were to take Dr McKusick's course, meet a few BSD people and make contacts within one of the BSD projects to start contributing. I was successful on all these fronts, so for me this was mission accomplished. Another first!
autoconf/clang (No) Fun and Games (https://undeadly.org/cgi?action=article;sid=20170930133438)
Robert Nagy (robert@) wrote in with a fascinating story of hunting down a recent problem with ports:
You might have been noticing the amount of commits to ports regarding autoconf and nested functions and asking yourself… what the hell is this all about?
I was hanging out at my friend Antoine (ajacoutot@)'s place just before EuroBSDCon 2017 started and we were having drinks and he told me that there is this weird bug where Gnome hangs completely after just a couple of seconds of usage and the gnome-shell process just sits in the fsleep state. This started to happen at the time when inteldrm(4) was updated, the default compiler was switched to clang(1) and futexes were turned on by default.
The next day we started to have a look at the issue and since the process was hanging in fsleep, it seemed clear that the cause must be futexes, so we had to start bisecting the base system, which resulted in random success and failure. In the end we figured out that it is neither futex nor inteldrm(4) related, so the only thing that was left is the switch to clang.
Now the problem is that we have to figure out what part of the system needs to be build with clang to trigger this issue, so we kept on going and systematically recompiled the base system with gcc until everything was ruled out … and it kept on hanging.
We were drunk and angry that now we have to go and check hundreds of ports because gnome is not a small standalone port, so between two bottles of wine a build VM was fired up to do a package build with gcc, because manually building all the dependencies would just take too long and we had spent almost two days on this already.
Next day ~200 packages were available to bisect and figure out what's going on. After a couple of tries it turned out that the hang is being caused by the gtk+3 package, which is bad since almost everything is using gtk+3. Now it was time to figure out what file the gtk+3 source being built by clang is causing the issue. (Compiler optimizations were ruled out already at this point.) So another set of bisecting happened, building each subdirectory of gtk+3 with clang and waiting for the hang to manifest … and it did not. What the $f?
Okay so something else is going on and maybe the configure script of gtk+3 is doing something weird with different compilers, so I quickly did two configure runs with gcc and clang and simply diff'd the two directories. Snippets from the diff:
-GDKHIDDENVISIBILITYCFLAGS = -fvisibility=hidden
GDKHIDDENVISIBILITYCFLAGS =
-ltcvprogcompilerrttiexceptions=no
ltcvprogcompilerrttiexceptions=yes
-#define GDKEXTERN attribute((visibility("default"))) extern
-ltprogcompilernobuiltinflag=' -fno-builtin'
+ltprogcompilernobuiltinflag=' -fno-builtin -fno-rtti -fno-exceptions'
Okay, okay that's something, but wait … clang has symbol visibility support so what is going on again? Let's take a peek at config.log:
configure:29137: checking for -fvisibility=hidden compiler flag
configure:29150: cc -c -fvisibility=hidden -I/usr/local/include -I/usr/X11R6/include conftest.c >&5
conftest.c:82:17: error: function definition is not allowed here
int main (void) { return 0; }
^
1 error generated.
Okay that's clearly an error but why exactly? autoconf basically generates a huge shell script that will check for whatever you throw at it by creating a file called conftest.c and putting chunks of code into it and then trying to compile it. In this case the relevant part of the code was:
| int
| main ()
| {
| int main (void) { return 0; }
| ;
| return 0;
| }
That is a nested function declaration which is a GNU extension and it is not supported by clang, but that's okay, the question is why the hell would you use nested functions to check for simple compiler flags. The next step was to go and check what is going on in configure.ac to see how the configure script is generated. In the gtk+3 case the following snippet is used:
AC_MSG_CHECKING([for -fvisibility=hidden compiler flag])
ACTRYCOMPILE([], [int main (void) { return 0; }],
ACMSGRESULT(yes)
enablefvisibilityhidden=yes,
ACMSGRESULT(no)
enablefvisibilityhidden=no)
According to the autoconf manual the ACTRYCOMPILE macro accepts the following parameters:
That clearly states that a function body has to be specified because the function definition is already provided automatically, so doing ACTRYCOMPILE([], [int main (void) { return 0;}], instead of ACTRYCOMPILE([],[] will result in a nested function declaration, which will work just fine with gcc, even though the autoconf usage is wrong.
After fixing the autoconf macro in gtk+3 and rebuilding the complete port from scratch with clang, the hang completely went away as the proper CFLAGS and LDFLAGS were picked up by autoconf for the build.
At this point we realized that most of the ports tree uses autoconf so this issue might be a lot bigger than we thought, so I asked sthen@ to do a grep on the ports object directory and just search for "function definition is not allowed here", which resulted in about ~60 additional ports affected.
Out of the list of ports there were only two false positive matches. These were actually trying to test whether the compiler supports nested functions. The rest were a combination of several autoconf macros used in a wrong way, e.g: ACTRYCOMPILE, ACTRYLINK. Most of them were fixable by just removing the extra function declaration or by switching to other autoconf macros like ACLANGSOURCE where you can actually declare your own functions if need be.
The conclusion is that this issue was a combination of people not reading documentation and just copy/pasting autoconf snippets, instead of reading their documentation and using the macros in the way they were intended, and the fact that switching to a new compiler is never easy and bugs or undefined behaviour are always lurking in the dark.
Thanks to everyone who helped fixing all the ports up this quickly! Hopefully all of the changes can be merged upstream, so that others can benefit as well.
Interview - David Carlier - @devnexen (https://twitter.com/devnexen)
Software Engineer at Afilias
***
News Roundup
Setting up OpenBSD's LDAP Server (ldapd) with StartTLS and SASL (http://blog.databasepatterns.com/2017/08/setting-up-openbsds-ldap-server-ldapd.html)
A tutorial on setting up OpenBSD’s native LDAP server with TLS encryption and SASL authentication
OpenBSD has its own LDAP server, ldapd. Here's how to configure it for use with StartTLS and SASL authentication
Create a certificate (acme-client anyone?)
Create a basic config file
listen on em0 tls certificate ldapserver
This will listen on the em0 interface with tls using the certificate called ldapserver.crt / ldapserver.key
Validate the configuration:
/usr/sbin/ldapd -n
Enable and start the service:
rcctl enable ldapd
rcctl start ldapd
On the client machine:
pkg_add openldap-client
Copy the certificate to /etc/ssl/trusted.crt
Add this line to /etc/openldap/ldap.conf
TLS_CACERT /etc/ssl/trusted.crt
Enable and start the service
rcctl enable saslauthd
rcctl start saslauthd
Connect to ldapd (-ZZ means force TLS, use -H to specify URI):
ldapsearch -H ldap://ldapserver -ZZ
FreeBSD Picks Up Support for ZFS Channel Programs in -current (https://svnweb.freebsd.org/base?view=revision&revision=324163)
ZFS channel programs (ZCP) adds support for performing compound ZFS administrative actions via Lua scripts in a sandboxed environment (with time and memory limits).
This initial commit includes both base support for running ZCP scripts, and a small initial library of API calls which support getting properties and listing, destroying, and promoting datasets.
Testing: in addition to the included unit tests, channel programs have been in use at Delphix for several months for batch destroying filesystems.
Take a simple task as an example: Create a snapshot, then set a property on that snapshot. In the traditional system for this, when you issue the snapshot command, that closes the currently open transaction group (say #100), and opens a new one, #101. While #100 is being written to disk, other writes are accumulated in #101. Once #100 is flushed to disk, the ‘zfs snapshot’ command returns. You can then issue the ‘zfs set’ command. This actually ends up going into transaction group #102. Each administrative action needs to wait for the transaction group to flush, which under heavy loads could take multiple seconds. Now if you want to create AND set, you need to wait for two or three transaction groups. Meanwhile, during transaction group #101, the snapshot existed without the property set, which could cause all kinds of side effects.
ZFS Channel programs solves this by allowing you to perform a small scripted set of actions as a single atomic operation.
In Delphix’s appliance, they often needed to do as many as 15 operations together, which might take multiple minutes. Now with channel programs it is much faster, far safer, and has fewer chances of side effects
BSDCan 2017 - Matt Ahrens: Building products based on OpenZFS, using channel programs -- Video Soon (http://www.bsdcan.org/2017/schedule/events/854.en.html)
Software Is About Storytelling (http://bravenewgeek.com/software-is-about-storytelling/)
Tyler Treat writes on the brave new geek blog:
Software engineering is more a practice in archeology than it is in building. As an industry, we undervalue storytelling and focus too much on artifacts and tools and deliverables. How many times have you been left scratching your head while looking at a piece of code, system, or process? It’s the story, the legacy left behind by that artifact, that is just as important—if not more—than the artifact itself.
And I don’t mean what’s in the version control history—that’s often useless. I mean the real, human story behind something. Artifacts, whether that’s code or tools or something else entirely, are not just snapshots in time. They’re the result of a series of decisions, discussions, mistakes, corrections, problems, constraints, and so on. They’re the product of the engineering process, but the problem is they usually don’t capture that process in its entirety. They rarely capture it at all. They commonly end up being nothing but a snapshot in time.
It’s often the sign of an inexperienced engineer when someone looks at something and says, “this is stupid” or “why are they using X instead of Y?” They’re ignoring the context, the fact that circumstances may have been different. There is a story that led up to that point, a reason for why things are the way they are. If you’re lucky, the people involved are still around. Unfortunately, this is not typically the case. And so it’s not necessarily the poor engineer’s fault for wondering these things. Their predecessors haven’t done enough to make that story discoverable and share that context.
I worked at a company that built a homegrown container PaaS on ECS. Doing that today would be insane with the plethora of container solutions available now. “Why aren’t you using Kubernetes?” Well, four years ago when we started, Kubernetes didn’t exist. Even Docker was just in its infancy. And it’s not exactly a flick of a switch to move multiple production environments to a new container runtime, not to mention the politicking with leadership to convince them it’s worth it to not ship any new code for the next quarter as we rearchitect our entire platform. Oh, and now the people behind the original solution are no longer with the company. Good luck! And this is on the timescale of about five years. That’s maybe like one generation of engineers at the company at most—nothing compared to the decades or more software usually lives (an interesting observation is that timescale, I think, is proportional to the size of an organization). Don’t underestimate momentum, but also don’t underestimate changing circumstances, even on a small time horizon.
The point is, stop looking at technology in a vacuum. There are many facets to consider. Likewise, decisions are not made in a vacuum. Part of this is just being an empathetic engineer. The corollary to this is you don’t need to adopt every bleeding-edge tech that comes out to be successful, but the bigger point is software is about storytelling. The question you should be asking is how does your organization tell those stories? Are you deliberate or is it left to tribal knowledge and hearsay? Is it something you truly value and prioritize or simply a byproduct?
Documentation is good, but the trouble with documentation is it’s usually haphazard and stagnant. It’s also usually documentation of how and not why. Documenting intent can go a long way, and understanding the why is a good way to develop empathy. Code survives us. There’s a fantastic talk by Bryan Cantrill on oral tradition in software engineering (https://youtu.be/4PaWFYm0kEw) where he talks about this. People care about intent. Specifically, when you write software, people care what you think. As Bryan puts it, future generations of programmers want to understand your intent so they can abide by it, so we need to tell them what our intent was. We need to broadcast it. Good code comments are an example of this. They give you a narrative of not only what’s going on, but why. When we write software, we write it for future generations, and that’s the most underestimated thing in all of software. Documenting intent also allows you to document your values, and that allows the people who come after you to continue to uphold them.
Storytelling in software is important. Without it, software archeology is simply the study of puzzles created by time and neglect. When an organization doesn’t record its history, it’s bound to repeat the same mistakes. A company’s memory is comprised of its people, but the fact is people churn. Knowing how you got here often helps you with getting to where you want to be. Storytelling is how we transcend generational gaps and the inevitable changing of the old guard to the new guard in a maturing engineering organization. The same is true when we expand that to the entire industry. We’re too memoryless—shipping code and not looking back, discovering everything old that is new again, and simply not appreciating our lineage.
Beastie Bits
1st BSD Users Stockholm Meetup (https://www.meetup.com/en-US/BSD-Users-Stockholm/)
Absolute FreeBSD, 3rd Edition draft completed (https://blather.michaelwlucas.com/archives/3020)
Absolute FreeBSD, 3rd Edition Table of Contents (https://blather.michaelwlucas.com/archives/2995)
t2k17 Hackathon Report: My first time (Aaron Bieber) (https://undeadly.org/cgi?action=article;sid=20170824193521)
The release of pfSense 2.4.0 will be slightly delayed to apply patches for vulnerabilities in 3rd party packages that are part of pfSense (https://www.netgate.com/blog/no-plan-survives-contact-with-the-internet.html)
Feedback/Questions
Ben writes in that zrepl is in ports now (http://dpaste.com/1XMJYMH#wrap)
Peter asks us about Netflix on BSD (http://dpaste.com/334WY4T#wrap)
meka writes in about dhclient exiting (http://dpaste.com/3GSGKD3#wrap)
***
| |||
| 04 Sep 2013 | 1: BGP & BSD | 01:53:51 | |
We kick off the first episode with the latest BSD news, show you how to avoid intrusion detection systems and talk to Peter Hessler about BGP spam blacklists!
Headlines
Radeon KMS commited (https://lists.freebsd.org/pipermail/svn-src-head/2013-August/050931.html)
Committed by Jean-Sebastien Pedron
Brings kernel mode setting to -CURRENT, will be in 10.0-RELEASE (ETA 12/2013)
10-STABLE is expected to be branched in October, to begin the process of stabilizing development
Initial testing shows it works well
May be merged to 9.X, but due to changes to the VM subsystem this will require a lot of work, and is currently not a priority for the Radeon KMS developer
Still suffers from the syscons / KMS switcher issues, same as Intel video
More info: https://wiki.freebsd.org/AMD_GPU
***
VeriSign Embraces FreeBSD (http://www.eweek.com/enterprise-apps/verisign-embraces-open-source-freebsd-for-diversity/)
"BSD is quite literally at the very core foundation of what makes the Internet work"
Using BSD and Linux together provides reliability and diversity
Verisign gives back to the community, runs vBSDCon
"You get comfortable with something because it works well for your particular purposes and can find a good community that you can interact with. That all rang true for us with FreeBSD."
***
fetch/libfetch get a makeover (http://freshbsd.org/commit/freebsd/r253680)
Adds support for SSL certificate verification
Requires root ca bundle (security/rootcanss)
Still missing TLS SNI support (Server Name Indication, allows name based virtual hosts over SSL)
***
FreeBSD Foundation Semi-Annual Newsletter (http://www.freebsdfoundation.org/press/2013Jul-newsletter)
The FreeBSD Foundation took the 20th anniversary of FreeBSD as an opportunity to look at where the project is, and where it might want to go
The foundation sets out some basic goals that the project should strive towards:
Unify User Experience
“ensure that knowledge gained mastering one task translates to the next”
“if we do pay attention to consistency, not only will FreeBSD be easier to use, it will be easier to learn”
Design for Human and Programmatic Use
200 machines used to be considered a large deployment, with high density servers, blades, virtualization and the cloud, that is not so anymore
“the tools we provide for status reporting, configuration, and control of FreeBSD just do not scale or fail to provide the desired user experience”
“The FreeBSD of tomorrow needs to give programmability and human interaction equal weighting as requirements”
Embrace New Ways to Document FreeBSD
More ‘Getting Started’ sections in documentation
Link to external How-Tos and other documentation
“upgrade the cross-referencing and search tools built into FreeBSD, so FreeBSD, not an Internet search engine, is the best place to learn about FreeBSD”
Spring Fundraising Campaign, April 17 - May 31, raised a total of $219,806 from 12 organizations and 365 individual donors. In the same period last year we raised a total of $23,422 from 2 organizations and 53 individuals
Funds donated to the FreeBSD Foundation have been used on these projects recently:
Capsicum security-component framework
Transparent superpages support of the FreeBSD/ARM architecture
Expanded and faster IPv6
Native in-kernel iSCSI stack
Five New TCP Congestion Control Algorithms
Direct mapped I/O to avoid extra memory copies
Unified Extensible Firmware Interface (UEFI) boot environment
Porting FreeBSD to the Genesi Efika MX SmartBook laptop (ARM-based)
NAND Flash filesystem and storage stack
Funds were also used to sponsor a number of BSD focused conferences: BSDCan, EuroBSDCon, AsiaBSDCon, BSDDay, NYCBSDCon, vBSDCon, plus Vendor summits and Developer summits
It is important that the foundation receive donations from individuals, to maintain their tax exempt status in the USA. Even a donation of $5 helps make it clear that the FreeBSD Foundation is backed by a large community, not only a few vendors
Donate Today (http://www.freebsdfoundation.org/donate)
***
The place to B...SD
Ohio Linuxfest, Sept. 13-15, 2013 (http://ohiolinux.org/schedule)
Very BSD friendly
Kirk McKusick giving the keynote
BSD Certification on the 15th, all other stuff on the 14th
Multiple BSD talks
***
LinuxCon, Sept. 16-18, 2013 (http://events.linuxfoundation.org/events/linuxcon-north-america)
Dru Lavigne and Kris Moore will be manning a FreeBSD booth
Number of talks of interest to BSD users, including ZFS coop (http://linuxconcloudopenna2013.sched.org/event/b50b23f3ed3bd728fa0052b54021a2cc?iframe=yes&w=900&sidebar=yes&bg=no)
EuroBSDCon, Sept. 26-29, 2013 (http://2013.eurobsdcon.org/eurobsdcon-2013/talks/)
Tutorials on the 26 & 27th (plus private FreeBSD DevSummit)
43 talks spread over 3 tracks on the 28 & 29th
Keynote by Theo de Raadt
Hosted in the picturesque St. Julians Area, Malta (Hilton Conference Centre)
***
Interview - Peter Hessler - phessler@openbsd.org (mailto:phessler@openbsd.org) / @phessler (https://twitter.com/phessler)
Using BGP to distribute spam blacklists and whitelists
Tutorial
Using stunnel to hide your traffic from Deep Packet Inspection (http://www.bsdnow.tv/tutorials/stunnel)
News Roundup
NetBSD 6.1.1 released (https://blog.netbsd.org/tnf/entry/netbsd_6_1_1_released)
First security/bug fix update of the NetBSD 6.1 release branch
Fixes 4 security vulnerabilities
Adds 4 new sysctls to avoid IPv6 DoS attacks
Misc. other updates
***
Sudo Mastery (http://blather.michaelwlucas.com/archives/1792)
MWL is a well-known author of many BSD books
Also does SSH, networking, DNSSEC, etc.
Next book is about sudo, which comes from OpenBSD (did you know that?)
Available for preorder now at a discounted price
***
Documentation Infrastructure Enhancements (http://freebsdfoundation.blogspot.com/2013/08/new-funded-project-documentation.html)
Gábor Kövesdán has completed a funded project to improve the infrastructure behind the documentation project
Will upgrade documentation from DocBook 4.2 to DocBook 4.5 and at the same time migrate to proper XML tools.
DSSSL is an old and dead standard, which will not evolve any more.
DocBook 5.0 tree added
***
FreeBSD FIBs get new features (https://svnweb.freebsd.org/base?view=revision&revision=254943)
FIBs (as discussed earlier in the interview) are Forward Information Bases (technical term for a routing table)
The FreeBSD kernel can be compiled to allow you to maintain multiple FIBs, creating separate routing tables for different processes or jails
In r254943 ps(1) is extended to support a new column ‘fib’, to display which routing table a process is using
***
FreeNAS 9.1.0 and 9.1.1 released (http://www.ixsystems.com/resources/ix/news/ixsystems-announces-revolutionary-freenas-910-release.html)
Many improvements in nearly all areas, big upgrade
Based on FreeBSD 9-STABLE, lots of new ZFS features
Cherry picked some features from 10-CURRENT
New volume manager and easy to use plugin management system
9.1.1 released shortly thereafter to fix a few UI and plugin bugs
***
BSD licensed "patch" becomes default (http://freshbsd.org/commit/freebsd/r253689)
bsdpatch has become mature, does what GNU patch can do, but has a much better license
Approved by portmgr@ for use in ports
Added WITHGNUPATCH build option for people who still need it
***
| |||
| 01 Feb 2018 | 231: Unix Architecture Evolution | 01:24:56 | |
We cover an interview about Unix Architecture Evolution, another vBSDcon trip report, how to teach an old Unix about backspace, new NUMA support coming to FreeBSD, and stack pointer checking in OpenBSD.
This episode was brought to you by
Headlines
Unix Architecture Evolution from the 1970 PDP-7 to the 2017 FreeBSD (https://fosdem.org/2018/interviews/diomidis-spinellis/)
Q: Could you briefly introduce yourself?
I’m a professor of software engineering, a programmer at heart, and a technology author. Currently I’m also the editor in chief of the IEEE Software magazine. I recently published the book Effective Debugging, where I detail 66 ways to debug software and systems.
Q: What will your talk be about, exactly?
I will describe how the architecture of the Unix operating system evolved over the past half century, starting from an unnamed system written in PDP-7 assembly language and ending with a modern FreeBSD system. My talk is based, first, on a GitHub repository where I tried to record the system’s history from 1970 until today and, second, on the evolution of documented facilities (user commands, system calls, library functions) across revisions. I will thus present the early system’s defining architectural features (layering, system calls, devices as files, an interpreter, and process management) and the important ones that followed in subsequent releases: the tree directory structure, user contributed code, I/O redirection, the shell as a user program, groups, pipes, scripting, and little languages.
Q: Why this topic?
Unix stands out as a major engineering breakthrough due to its exemplary design, its numerous technical contributions, its impact, its development model, and its widespread use. Furthermore, the design of the Unix programming environment has been characterized as one offering unusual simplicity, power, and elegance. Consequently, there are many lessons that we can learn by studying the evolution of the Unix architecture, which we can apply to the design of new systems. I often see modern systems that suffer from a bloat of architectural features and a lack of clear form on which functionality can be built. I believe that many of the modern Unix architecture defining features are excellent examples of what we should strive toward as system architects.
Q: What do you hope to accomplish by giving this talk? What do you expect?
I’d like FOSDEM attendees to leave the talk with their mind full with architectural features of timeless quality. I want them to realize that architectural elegance isn’t derived by piling design patterns and does not need to be expensive in terms of resources. Rather, beautiful architecture can be achieved on an extremely modest scale. Furthermore, I want attendees to appreciate the importance of adopting flexible conventions rather than rigid enforcement mechanisms. Finally, I want to demonstrate through examples that the open source culture was part of Unix from its earliest days.
Q: What are the most significant milestones in the development of Unix?
The architectural development of Unix follows a path of continuous evolution, albeit at a slowing pace, so I don’t see here the most important milestones. I would however define as significant milestones two key changes in the way Unix was developed. The first occurred in the late 1970s when significant activity shifted from a closely-knit team of researchers at the AT&T Bell Labs to the Computer Science Research Group in the University of California at Berkeley. This opened the system to academic contributions and growth through competitive research funding. The second took place in the late 1980s and the 1990s when Berkeley open-sourced the the code it had developed (by that time a large percentage of the system) and enthusiasts built on it to create complete open source operating system distributions: 386BSD, and then FreeBSD, NetBSD, OpenBSD, and others.
Q: In which areas has the development of Unix stalled?
The data I will show demonstrate that there were in the past some long periods where the number of C library functions and system calls remained mostly stable. Nowadays there is significant growth in the number of all documented facilities with the exception of file formats. I’m looking forward to a discussion regarding the meaning of these growth patterns in the Q&A session after the talk.
Q: What are the core features that still link the 1970 PDP-7 system to the latest FreeBSD 11.1 release, almost half a century apart?
Over the past half-century the Unix system has grown by four orders of magnitude from a few thousand lines of code to many millions. Nevertheless, looking at a 1970s architecture diagram and a current one reveals that the initial architectural blocks are still with us today. Furthermore, most system calls, user programs, and C library functions of that era have survived until today with essentially similar functionality. I’ve even found in modern FreeBSD some lines of code that have survived unchanged for 40 years.
Q: Can we still add innovative changes to operating systems like FreeBSD without breaking the ‘Unix philosophy’? Will there be a moment where FreeBSD isn’t recognizable anymore as a descendant of the 1970 PDP-7 system?
There’s a saying that “form liberates”. So having available a time-tested form for developing operating system functionality allows you to innovate in areas that matter rather than reinventing the wheel.
Such concepts include having commands act as a filter, providing manual pages with a consistent structure, supplying build information in the form of a Makefile, installing files in a well-defined directory hierarchy, implementing filesystems with an standardized object-oriented interface, and packaging reusable functions as a library. Within this framework there’s ample space for both incremental additions (think of jq, the JSON query command) and radical innovations (consider the Solaris-derived ZFS and dtrace functionality). For this reason I think that BSD and Linux systems will always be recognizable as direct or intellectual descendants of the 1970s Research Unix editions.
Q: Have you enjoyed previous FOSDEM editions?
Immensely! As an academic I need to attend many scientific conferences and meetings in order to present research results and interact with colleagues. This means too much time spent traveling and away from home, and a limited number of conferences I’m in the end able to attend. Nevertheless, attending FOSDEM is an easy decision due to the world-changing nature of its theme, the breadth of the topics presented, the participants’ enthusiasm and energy, as well as the exemplary, very efficient conference organization.
Another vBSDCon trip report we just found (https://www.weaponizedawesome.com/blog/?cat=53)
We just got tipped about another trip report from vBSDCon, this time from one of the first time speakers: W. Dean Freeman
Recently I had the honor of co-presenting on the internals of FreeBSD’s Kernel RNG with John-Mark Gurney at the 3rd biennial vBSDCon, hosted in Reston, VA hosted by Verisign.
I’ve been in and out of the FreeBSD community for about 20 years. As I’ve mentioned on here before, my first Unix encounter was FreeBSD 2.2.8 when I was in the 7th or 8th grade. However, for all that time I’ve never managed to get out to any of the cons. I’ve been to one or two BUG meetings and I’ve met some folks from IRC before, but nothing like this.
A BSD conference is a very different experience than anything else out there. You have to try it, it is the only way to truly understand it.
I’d also not had to do a stand-up presentation really since college before this. So, my first BSD con and my first time presenting rolled into one made for an interesting experience.
See, he didn’t say terrifying. It went very well. You should totally submit a talk for the next conference, even if it is your first.
That said, it was amazing and invigorating experience. I got to meet a few big names in the FreeBSD community, discuss projects, ideas for FreeBSD, etc. I did seem to spend an unusual amount of time talking about FIPS and Common Criteria with folks, but to me that’s a good sign and indicative that there is interest in working to close gaps between FreeBSD and the current requirements so that we can start getting FreeBSD and more BSD-based products into the government and start whittling away the domination of Linux (especially since Oracle has cut Solaris, SPARC and the ZFS storage appliance business units).
There is nothing that can match the high bandwidth interchange of ideas in person. The internet has made all kinds of communication possible, and we use it all the time, but every once in a while, getting together in person is hugely valuable.
Dean then went on to list some of the talks he found most valuable, including DTrace, Capsicum, bhyve, *BSD security tools, and Paul Vixie’s talk about gets()
I think the talk that really had the biggest impact on me, however, was Kyle Kneisl’s talk on BSD community dynamics. One of the key points he asked was whether the things that drew us to the BSD community in the first place would be able to happen today. Obviously, I’m not a 12 or 13 year old kid anymore, but it really got me thinking. That, combined with getting face time with people I’d previously only known as screen names has recently drawn me back into participating in IRC and rejoining mailing lists (wdf on freenode. be on the lookout!)
Then Dean covered some thoughts on his own talk:
JMG and my talk seems to have been well received, with people paying lots of attention. I don’t know what a typical number of questions is for one of these things, but on day one there weren’t that many questions. We got about 5 during our question time and spent most of the rest of the day fielding questions from interested attendees. Getting a “great talk!” from GNN after coming down from the stage was probably one of the major highlights for me.
I remember my first solo talk, and GNN asking the right question in the middle to get me to explain a part of it I had missed. It was very helpful.
I think key to the interest in our presentation was that JMG did a good job framing a very complicated topic’s importance in terms everyone could understand. It also helped that we got to drop some serious truth bombs.
Final Thoughts:
I met a lot of folks in person for the first time, and met some people I’d never known online before. It was a great community and I’m glad I got a chance to expand my network.
Verisign were excellent hosts and they took good care of both speakers (covering airfare, rooms, etc.) and also conference attendees at large. The dinners that they hosted were quite good as well.
I’m definitely interested in attending vBSDCon again and now that I’ve had a taste of meeting IRL with the community on scale of more than a handful, I have every intention of finally making it to BSDCan next year (I’d said it in 2017, but then moved to Texas for a new job and it wasn’t going to be practical). This year for sure, though!
Teaching an Almost 40-year Old UNIX about Backspace (https://virtuallyfun.com/2018/01/17/teaching_an_almost_40-year_old_unix_about_backspace/)
Introduction
I have been messing with the UNIX® operating system, Seventh Edition (commonly known as UNIX V7 or just V7) for a while now. V7 dates from 1979, so it’s about 40 years old at this point. The last post was on V7/x86, but since I’ve run into various issues with it, I moved on to a proper installation of V7 on SIMH. The Internet has some really good resources on installing V7 in SIMH. Thus, I set out on my own journey on installing and using V7 a while ago, but that was remarkably uneventful.
One convenience that I have been dearly missing since the switch from V7/x86 is a functioning backspace key. There seem to be multiple different definitions of backspace:
BS, as in ASCII character 8 (010, 0x08, also represented as ^H), and
DEL, as in ASCII character 127 (0177, 0x7F, also represented as ^?).
V7 does not accept either for input by default. Instead, # is used as the erase character and @ is used as the kill character. These defaults have been there since UNIX V1. In fact, they have been “there” since Multics, where they got chosen seemingly arbitrarily. The erase character erases the character before it. The kill character kills (deletes) the whole line. For example, “ba##gooo#d” would be interpreted as “good” and “bad line@good line” would be interpreted as “good line”.
There is some debate on whether BS or DEL is the correct character for terminals to send when the user presses the backspace key. However, most programs have settled on DEL today. tmux forces DEL, even if the terminal emulator sends BS, so simply changing my terminal to send BS was not an option. The change from the defaults outlined here to today’s modern-day defaults occurred between 4.1BSD and 4.2BSD. enf on Hacker News has written a nice overview of the various conventions
Getting the Diff
For future generations as well as myself when I inevitably majorly break this installation of V7, I wanted to make a diff. However, my V7 is installed in SIMH. I am not a very intelligent man, I didn’t keep backup copies of the files I’d changed. Getting data out of this emulated machine is an exercise in frustration.
In the end, I printed everything on screen using cat(1) and copied that out. Then I performed a manual diff against the original source code tree because tabs got converted to spaces in the process. Then I applied the changes to clean copies that did have the tabs. And finally, I actually invoked diff(1).
Closing Thoughts
Figuring all this out took me a few days. Penetrating how the system is put together was surprisingly fairly hard at first, but then the difficulty curve eased up. It was an interesting exercise in some kind of “reverse engineering” and I definitely learned something about tty handling. I was, however, not pleased with using ed(1), even if I do know the basics. vi(1) is a blessing that I did not appreciate enough until recently. Had I also been unable to access recursive grep(1) on my host and scroll through the code, I would’ve probably given up. Writing UNIX under those kinds of editing conditions is an amazing feat. I have nothing but the greatest respect for software developers of those days.
News Roundup
New NUMA support coming to FreeBSD CURRENT (https://lists.freebsd.org/pipermail/freebsd-current/2018-January/068145.html)
Hello folks,
I am working on merging improved NUMA support with policy implemented by cpuset(2) over the next week. This work has been supported by Dell/EMC's Isilon product division and Netflix. You can see some discussion of these changes here:
https://reviews.freebsd.org/D13403
https://reviews.freebsd.org/D13289
https://reviews.freebsd.org/D13545
The work has been done in user/jeff/numa if you want to look at svn history or experiment with the branch. It has been tested by Peter Holm on i386 and amd64 and it has been verified to work on arm at various points.
We are working towards compatibility with libnuma and linux mbind. These commits will bring in improved support for NUMA in the kernel. There are new domain specific allocation functions available to kernel for UMA, malloc, kmem, and vmpage*. busdmamem consumers will automatically be placed in the correct domain, bringing automatic improvements to some
device performance.
cpuset will be able to constrains processes, groups of processes, jails, etc. to subsets of the system memory domains, just as it can with sets of cpus. It can set default policy for any of the above. Threads can use cpusets to set policy that specifies a subset of their visible domains.
Available policies are first-touch (local in linux terms), round-robin (similar to linux interleave), and preferred. For now, the default is round-robin. You can achieve a fixed domain policy by using round-robin with a bitmask of a single domain. As the scheduler and VM become more
sophisticated we may switch the default to first-touch as linux does.
Currently these features are enabled with VMNUMAALLOC and MAXMEMDOM. It will eventually be NUMA/MAXMEMDOM to match SMP/MAXCPU. The current NUMA syscalls and VMNUMAALLOC code was 'experimental' and will be deprecated. numactl will continue to be supported although cpuset should be preferred going forward as it supports the full feature set of the new API.
Thank you for your patience as I deal with the inevitable fallout of such sweeping changes. If you do have bugs, please file them in bugzilla, or reach out to me directly. I don't always have time to catch up on all of my mailing list mail and regretfully things slip through the cracks when
they are not addressed directly to me.
Thanks,
Jeff
Stack pointer checking – OpenBSD (https://marc.info/?l=openbsd-tech&m=151572838911297&w=2)
Stefan (stefan@) and I have been working for a few months on this diff, with help from a few others.
At every trap and system call, it checks if the stack-pointer is on a page that is marked MAPSTACK. execve() is changed to create such mappings for the process stack. Also, libpthread is taught the new MAPSTACK flag to use with mmap().
There is no corresponding system call which can set MAP_FLAG on an existing page, you can only set the flag by mapping new memory into place. That is a piece of the security model.
The purpose of this change is to twart stack pivots, which apparently have gained some popularity in JIT ROP attacks. It makes it difficult to place the ROP stack in regular data memory, and then perform a system call from it. Workarounds are cumbersome, increasing the need for far more gadgetry. But also the trap case -- if any memory experiences a demand page fault, the same check will occur and potentially also kill the process.
We have experimented a little with performing this check during device interrupts, but there are some locking concerns and performance may then become a concern. It'll be best to gain experience from handle of syncronous trap cases first.
chrome and other applications I use run fine!
I'm asking for some feedback to discover what ports this breaks, we'd like to know. Those would be ports which try to (unconventionally) create their stacks in malloc()'d memory or inside another Data structure. Most of them are probably easily fixed ...
Qt 5.9 on FreeBSD (https://euroquis.nl/bobulate/?p=1768)
Tobias and Raphael have spent the past month or so hammering on the Qt 5.9 branch, which has (finally!) landed in the official FreeBSD ports tree. This brings FreeBSD back up-to-date with current Qt releases and, more importantly, up-to-date with the Qt release KDE software is increasingly expecting. With Qt 5.9, the Elisa music player works, for instance (where it has run-time errors with Qt 5.7, even if it compiles). The KDE-FreeBSD CI system has had Qt 5.9 for some time already, but that was hand-compiled and jimmied into the system, rather than being a “proper” ports build.
The new Qt version uses a new build system, which is one of the things that really slowed us down from a packaging perspective. Some modules have been reshuffled in the process. Some applications depending on Qt internal-private headers have been fixed along the way. The Telegram desktop client continues to be a pain in the butt that way.
Following on from Qt 5.9 there has been some work in getting ready for Clang 6 support; in general the KDE and Qt stack is clean and modern C++, so it’s more infrastructural tweaks than fixing code. Outside of our silo, I still see lots of wonky C++ code being fixed and plenty of confusion between pointers and integers and strings and chars and .. ugh. Speaking of ugh, I’m still planning to clean up Qt4 on ARM aarch64 for FreeBSD; this boils down to stealing suitable qatomic implementations from Arch Linux.
For regular users of Qt applications on FreeBSD, there should be few to no changes required outside the regular upgrade cycle. For KDE Plasma users, note that development of the ports has changed branches; as we get closer to actually landing modern KDE bits, things have been renamed and reshuffled and mulled over so often that the old plasma5 branch wasn’t really right anymore. The kde5-import branch is where it’s at nowadays, and the instructions are the same: the x11/kde5 metaport will give you all the KDE Frameworks 5, KDE Plasma Desktop and modern KDE Applications you need.
Adding IPv6 to an Nginx website on FreeBSD / FreshPorts (https://dan.langille.org/2018/01/13/adding-ipv6-to-an-nginx-website-on-freebsd-freshports/)
FreshPorts recently moved to an IPv6-capable server but until today, that capability has not been utilized.
There were a number of things I had to configure, but this will not necessarily be an exhaustive list for you to follow. Some steps might be missing, and it might not apply to your situation.
All of this took about 3 hours.
We are using:
FreeBSD 11.1
Bind 9.9.11
nginx 1.12.2
Fallout
I expect some monitoring fallout from this change. I suspect some of my monitoring assumes IP4 and now that IPv6 is available, I need to monitor both IP addresses.
ZFS on TrueOS: Why We Love OpenZFS (https://www.trueos.org/blog/zfs-trueos-love-openzfs/)
TrueOS was the first desktop operating system to fully implement the OpenZFS (Zettabyte File System or ZFS for short) enterprise file system in a stable production environment. To fully understand why we love ZFS, we will look back to the early days of TrueOS (formerly PC-BSD). The development team had been using the UFS file system in TrueOS because of its solid track record with FreeBSD-based computer systems and its ability to check file consistency with the built-in check utility fsck.
However, as computing demands increased, problems began to surface. Slow fsck file verification on large file systems, slow replication speeds, and inconsistency in data integrity while using UFS logging / journaling began to hinder users. It quickly became apparent that TrueOS users would need a file system that scales with evolving enterprise storage needs, offers the best data protection, and works just as well on a hobbyist system or desktop computer.
Kris Moore, the founder of the TrueOS project, first heard about OpenZFS in 2007 from chatter on the FreeBSD mailing lists. In 2008, the TrueOS development team was thrilled to learn that the FreeBSD Project had ported ZFS. At the time, ZFS was still unproven as a graphical desktop solution, but Kris saw a perfect opportunity to offer ZFS as a cutting-edge file system option in the TrueOS installer, allowing the TrueOS project to act as an indicator of how OpenZFS would fair in real-world production use.
The team was blown away by the reception and quality of OpenZFS on FreeBSD-based systems. By its nature, ZFS is a copy-on-write (CoW) file system that won’t move a block of data until it both writes the data and verifies its integrity. This is very different from most other file systems in use today. ZFS is able to assure that data stays consistent between writes by automatically comparing write checksums, which mitigates bit rot. ZFS also comes with native RaidZ functionality that allows for enterprise data management and redundancy without the need for expensive traditional RAID cards. ZFS snapshots allow for system configuration backups in a split-second. You read that right. TrueOS can backup or restore snapshots in less than a second using the ZFS file system.
Given these advantages, the TrueOS team decided to use ZFS as its exclusive file system starting in 2013, and we haven’t looked back since. ZFS offers TrueOS users the stable workstation experience they want, while simultaneously scaling to meet the increasing demands of the enterprise storage market. TrueOS users are frequently commenting on how easy it is to use ZFS snapshots with our built-in snapshot utility. This allows users the freedom to experiment with their system knowing they can restore it in seconds if anything goes wrong. If you haven’t had a chance to try ZFS with TrueOS, browse to our download page and make sure to grab a copy of TrueOS. You’ll be blown away by the ease of use, data protection functionality, and incredible flexibility of RaidZ.
Beastie Bits
Source Code Podcast Interview with Michael W Lucas (https://blather.michaelwlucas.com/archives/3099)
Operating System of the Year 2017: NetBSD Third place (https://w3techs.com/blog/entry/web_technologies_of_the_year_2017)
OPNsense 18.1-RC1 released (https://opnsense.org/opnsense-18-1-rc1-released/)
Personal OpenBSD Wiki Notes (https://balu-wiki.readthedocs.io/en/latest/security/openbsd.html)
BSD section can use some contribution (https://guide.freecodecamp.org/bsd-os/)
The Third Research Edition Unix Programmer's Manual (now available in PDF) (https://github.com/dspinellis/unix-v3man)
Feedback/Questions
Alex - my first freebsd bug (http://dpaste.com/3DSV7BC#wrap)
John - Suggested Speakers (http://dpaste.com/2QFR4MT#wrap)
Todd - Two questions (http://dpaste.com/2FQ450Q#wrap)
Matthew - CentOS to FreeBSD (http://dpaste.com/3KA29E0#wrap)
Brian - Brian - openbsd 6.2 and enlightenment .17 (http://dpaste.com/24DYF1J#wrap)
***
| |||
| 01 Feb 2017 | 179: The Wayland Machine | 00:56:54 | |
This week on BSDNow, we’re going to be leading off with the latest news about Wayland and Xorg support on FreeBSD, then a look at OpenBSD ARM64
This episode was brought to you by
Headlines
Wayland is now in the FreeBSD Ports tree (https://svnweb.freebsd.org/ports?view=revision&revision=432406)
This commit brings Wayland, the new windowing system, into the FreeBSD ports tree
“This port was first created by Koop Mast (kwm@) then updated and improved by Johannes Lundberg”
“Wayland is intended as a simpler replacement for X, easier to develop and maintain. GNOME and KDE are expected to be ported to it.”
Wayland is designed for desktop and laptop use, rather than X, which was designed for use over the network, where clients were not powerful enough to run the applications locally.
“Wayland is a protocol for a compositor to talk to its clients as well as a C library implementation of that protocol. The compositor can be a standalone display server running on Linux kernel modesetting and evdev input devices, an X application, or a wayland client itself. The clients can be traditional applications, X servers (rootless or fullscreen) or other display servers.”
“Please report bugs to the FreeBSD bugtracker!”
It is good to see this project progressing, as it seems in a few generations, high performance graphics drivers may only be actively developed for Wayland.
***
Call For Testing: xorg 1.18.4 and newer intel/ati DDX (https://lists.freebsd.org/pipermail/freebsd-x11/2017-January/018738.html)
Baptiste Daroussin, and the FreeBSD X11 team, have issued a call for testing for the upgrade to Xorg 1.18.4
Along with it comes newer ATI/AMD and Intel drivers
“Note that you will need to rebuild all the xf86-* packages to work with thatnewer xorg (hence the bump of the revision)”
“Do not expect newer gpu supported as this is not the kernel part”, it only provides the newer Xorg driver, not the kernel mode setting driver (this is a separate project)
“If you experience any issue with intel or radeon driver please try to use the new modesetting driver provided by xorg directly (note that fedora and debian recommend the use of the new driver instead of the ati/intel one)”
***
Error handling in C (http://www.tedunangst.com/flak/post/to-errno-or-to-error)
“Unlike other languages which have one preferred means of signalling an error, C is a multi error paradigm language. Error handling styles in C can be organized into one of several distinct styles, such as popular or correct. Some examples of each.”
“One very popular option is the classic unix style. -1 is returned to indicate an error.”
“Another option seen in the standard C library is NULL for errors.”
“The latter has the advantage that NULL is a false value, which makes it easier to write logical conditions. File descriptor 0 is valid (stdin) but false, while -1 is invalid but true.”
“And of course, there’s the worst of both worlds approach requiring a special sentinel that you’ll probably forget to use”
“Other unix functions, those that don’t need to return a file descriptor, stick to just 0 and -1”
“Of course, none of these functions reveal anything about the nature of the error. For that, you need to consult the errno on the side”
The article goes on to describe different ways of dealing with the issue, and return values.
There is also coverage of more complex examples and involve a context that might contain the error message
It is really interesting to see the differences, and the pitfalls of each approach
***
Fixing POSIX Filenames (http://www.dwheeler.com/essays/fixing-unix-linux-filenames.html)
“Traditionally, Unix/Linux/POSIX pathnames and filenames can be almost any sequence of bytes. A pathname lets you select a particular file, and may
include zero or more “/” characters. Each pathname component (separated by “/”) is a filename; filenames cannot contain “/”. Neither
filenames nor pathnames can contain the ASCII NUL character (\0), because that is the terminator.”
“This lack of limitations is flexible, but it also creates a legion of unnecessary problems. In particular, this lack of limitations makes it
unnecessarily difficult to write correct programs (enabling many security flaws). It also makes it impossible to consistently and accurately display filenames,
causes portability problems, and confuses users.”
“This article will try to convince you that adding some tiny limitations on legal Unix/Linux/POSIX filenames would be an improvement. Many programs
already presume these limitations, the POSIX standard already permits such limitations, and many Unix/Linux filesystems already embed such limitations — so
it’d be better to make these (reasonable) assumptions true in the first place. This article will discuss, in particular, the three biggest problems: control
characters in filenames (including newline, tab, and escape), leading dashes in filenames, and the lack of a standard character encoding scheme (instead of
using UTF-8). These three problems impact programs written in any language on Unix/Linux/POSIX system. There are other problems, of course. Spaces in filenames
can cause problems; it’s probably hopeless to ban them outright, but resolving some of the other issues will simplify handling spaces in filenames. For
example, when using a Bourne shell, you can use an IFS trick (using IFS=printf '\n\t') to eliminate some problems with spaces. Similarly, special
metacharacters in filenames cause some problems; I suspect few if any metacharacters could be forbidden on all POSIX systems, but it’d be great if
administrators could locally configure systems so that they could prevent or escape such filenames when they want to. I then discuss some other tricks that can
help.”
“After limiting filenames slightly, creating completely-correct programs is much easier, and some vulnerabilities in existing programs disappear. This
article then notes some others’ opinions; I knew that some people wouldn’t agree with me, but I’m heartened that many do agree that something should
be done. Finally, I briefly discuss some methods for solving this long-term; these include forbidding creation of such names (hiding them if they already exist
on the underlying filesystem), implementing escaping mechanisms, or changing how tools work so that these are no longer problems (e.g., when globbing/scanning,
have the libraries prefix “./” to any filename beginning with “-”). Solving this is not easy, and I suspect that several solutions will be
needed. In fact, this paper became long over time because I kept finding new problems that needed explaining (new “worms under the rocks”). If I’ve
convinced you that this needs improving, I’d like your help in figuring out how to best do it!”
“Filename problems affect programs written in any programming language. However, they can be especially tricky to deal with when using Bourne shells
(including bash and dash). If you just want to write shell programs that can handle filenames correctly, you should see the short companion article Filenames
and Pathnames in Shell: How to do it correctly (http://www.dwheeler.com/essays/filenames-in-shell.html).”
Imagine that you don’t know Unix/Linux/POSIX (I presume you really do), and that you’re trying to do some simple tasks. For our purposes we will
create simple scripts on the command line (using a Bourne shell) for these tasks, though many of the underlying problems affect any program. For example,
let’s try to print out the contents of all files in the current directory, putting the contents into a file in the parent directory:
cat * > ../collection # WRONG
cat ./* > ../collection # CORRECT
cat find . -type f > ../collection # WRONG
( set -f ; for file in find . -type f ; do # WRONG
cat "$file"
done ) > ../collection
( find . -type f | xargs cat ) > ../collection # WRONG, WAY WRONG
Just think about trying to remove a file named: -rf /
***
News Roundup
OpenBSD ARM64 (https://www.openbsd.org/arm64.html)
A new page has appeared on the OpenBSD website, offering images for ARM64
“The current target platforms are the Pine64 and the Raspberry Pi 3.”
“OpenBSD/arm64 bundles various platforms sharing the 64-bit ARM architecture. Due to the fact that there are many System on a Chips (SoC) around, OpenBSD/arm64 differentiates between various SoCs and may have a different level of support between them”
The page contains a list of the devices that are supported, and which components have working drivers
At the time of recording, the link to download the snapshots did not work yet, but by time this airs a week from now, it should be working.
***
The design of Chacha20 (http://loup-vaillant.fr/tutorials/chacha20-design)
Seems like every few episodes we end up discussing Ciphers (With their o-so amusing naming) and today is no exception.
We have a great writeup on the D & I of the ‘chacha20’ cipher written by “Loup Vaillant”
First of all, is this story for you? Maybe the summary will help make that call:
“Quick summary: Chacha20 is ARX-based hash function, keyed, running in counter mode. It embodies the idea that one can use a hash function to encrypt data.”
If your eyes didn’t glaze over, then you are cleared to proceed.
Chacha20 is built around stream ciphers:
While Chacha20 is mainly used for encryption, its core is a pseudo-random number generator. The cipher text is obtained by XOR'ing the plain text with a pseudo-random stream:
ciphertext = plaintext XOR chacha_stream(key, nonce)
Provided you never use the same nonce with the same key twice, you can treat that stream as a one time pad. This makes it very simple: unlike block ciphers, you don't have to worry about padding, and decryption is the same operation as encryption:
plaintext = ciphertext XOR chacha_stream(key, nonce)
Now we just have to get that stream.
The idea that the streams can mimic the concept of a one-time pad does make chacha20 very attractive, even to a non-crypto guy such as myself.
From here the article goes into depth on how the cipher scrambles 512bit blocks using the quarter-round method (A forth of a block or 4 32bit numbers)
Some ascii art is used here to help visualize how this done, in the quarter round-phase, then to the complete block as the 4 quarters are run in parallel over the entire 512 bit block.
From here the article goes more into depth, looking at the complete chacha block, and the importance of a seemingly unnecessary 32byte constant (Hint: it’s really important)
If crypto is something you find fascinating, you’ll want to make sure you give this one a full read-through.
***
CyberChef - Coming to a FreeBSD Ports tree near you (https://twitter.com/DLangille/status/823915729430913025)
Dan Langille tweets that he will be creating a port of GCHQ’s CyberChef tool
“CyberChef is a simple, intuitive web app for carrying out all manner of "cyber" operations within a web browser. These operations include creating hexdumps, simple encoding like XOR or Base64, more complex encryption like AES, DES and Blowfish, data compression and decompression, calculating hashes and checksums, IPv6 and X.509 parsing, and much more.”
“The tool is designed to enable both technical and non-technical analysts to manipulate data in complex ways without having to deal with complex tools or algorithms. It was conceived, designed, built and incrementally improved by an analyst in their 10% innovation time over several years. Every effort has been made to structure the code in a readable and extendable format, however it should be noted that the analyst is not a professional developer and the code has not been peer-reviewed for compliance with a formal specification.”
Some handy functions, beyond stuff like base64 encoding:
Network Enumeration (CIDR to list of IPS) (https://gchq.github.io/CyberChef/?recipe=%5B%7B%22op%22%3A%22Parse%20IP%20range%22%2C%22args%22%3A%5Btrue%2Ctrue%2Cfalse%5D%7D%5D&input=MTcyLjIxLjAuMzIvMjcK)
Browser User Agent Parser (what browser is that, based on your HTTP logs)
XOR Brute Force: enter some XOR’d text, and try every possible key to find plaintext. Optionally give it a regex of known plaintext to find the right key.
Calculate the “Shannon Entropy” of the input (how random is this data)
It also has a number of built in regular expressions for common things, very useful
The project is up on github if you want to play with the code
***
Building Electron and VSCode in FreeBSD11 (https://gist.github.com/prash-wghats/89be1ee069d2acf23c289e9c606616e1)
A patch and set of instructions for building Electron and VSCode on FreeBSD
“Visual Studio Code is a source code editor developed by Microsoft for Windows, Linux and macOS. It includes support for debugging, embedded Git control, syntax highlighting, intelligent code completion, snippets, and code refactoring. It is also customizable, so users can change the editor's theme, keyboard shortcuts, and preferences. It is free and open-source, although the official download is under a proprietary license.”
“Visual Studio Code is based on Electron, a framework which is used to deploy Node.js applications for the desktop running on the Blink layout engine. Although it uses the Electron framework, the software is not a fork of Atom, it is actually based on Visual Studio Online's editor (codename "Monaco")”
It would be interesting to see official support for VSCode on FreeBSD
Has anyone tried VSCode on the FreeBSD Code base?
***
Beastie Bits
Soft Label Keys (http://roy.marples.name/blog/blog/soft-label-keys)
WPA1 (TKIP) disabled by default (OpenBSD) (https://www.mail-archive.com/source-changes@openbsd.org/msg84599.html)
Cool but obscure unix tools (https://kkovacs.eu/cool-but-obscure-unix-tools)
KDE Frameworks and Plasma on FreeBSD (http://euroquis.nl/bobulate/?p=1521)
Initiative to migrate OpenBSD mirrors to HTTPS (https://www.mail-archive.com/source-changes@openbsd.org/msg84904.html)
That moment you realize FreeBSD has got some Star Wars fans (http://i.imgur.com/dC7c1y4.png)
Pagelink (https://wiki.freebsd.org/PortsSubversionPrimer)
| |||
| 29 Jul 2015 | 100: Straight from the Src | 01:13:39 | |
We've finally reached a hundred episodes, and this week we'll be talking to Sebastian Wiedenroth about pkgsrc. Though originally a NetBSD project, now it runs pretty much everywhere, and he even runs a conference about it!
This episode was brought to you by
Headlines
Remote DoS in the TCP stack (https://blog.team-cymru.org/2015/07/another-day-another-patch/)
A pretty devious bug in the BSD network stack has been making its rounds for a while now, allowing remote attackers to exhaust the resources of a system with nothing more than TCP connections
While in the LAST_ACK state, which is one of the final stages of a connection's lifetime, the connection can get stuck and hang there indefinitely
This problem has a slightly confusing history that involves different fixes at different points in time from different people
Juniper originally discovered the bug and announced a fix (https://kb.juniper.net/InfoCenter/index?page=content&id=JSA10686) for their proprietary networking gear on June 8th
On June 29th, FreeBSD caught wind of it and fixed the bug in their -current branch (https://svnweb.freebsd.org/base/head/sys/netinet/tcp_output.c?view=patch&r1=284941&r2=284940&pathrev=284941), but did not issue a security notice or MFC the fix back to the -stable branches
On July 13th, two weeks later, OpenBSD fixed the issue (https://www.marc.info/?l=openbsd-cvs&m=143682919807388&w=2) in their -current branch with a slightly different patch, citing the FreeBSD revision from which the problem was found
Immediately afterwards, they merged it back to -stable and issued an errata notice (http://ftp.openbsd.org/pub/OpenBSD/patches/5.7/common/010_tcp_persist.patch.sig) for 5.7 and 5.6
On July 21st, three weeks after their original fix, FreeBSD committed yet another slightly different fix (https://svnweb.freebsd.org/base/head/sys/netinet/tcp_output.c?view=patch&r1=285777&r2=285776&pathrev=285777) and issued a security notice (https://lists.freebsd.org/pipermail/freebsd-announce/2015-July/001655.html) for the problem (which didn't include the first fix)
After the second fix from FreeBSD, OpenBSD gave them both another look and found their single fix to be sufficient, covering the timer issue in a more general way
NetBSD confirmed they were vulnerable too, and applied another completely different fix (http://cvsweb.netbsd.org/bsdweb.cgi/src/sys/netinet/tcp_output.c.diff?r1=1.183&r2=1.184&only_with_tag=MAIN) to -current on July 24th, but haven't released a security notice yet
DragonFly is also investigating the issue now to see if they're affected as well
***
c2k15 hackathon reports (http://undeadly.org/cgi?action=article&sid=20150721180312&mode=flat)
Reports from OpenBSD's latest hackathon (http://www.openbsd.org/hackathons.html), held in Calgary this time, are starting to roll in (there were over 40 devs there, so we might see a lot more of these)
The first one, from Ingo Schwarze, talks about some of the mandoc work he did at the event
He writes, "Did you ever look at a huge page in man, wanted to jump to the definition of a specific term - say, in ksh, to the definition of the "command" built-in command - and had to step through dozens of false positives with the less '/' and 'n' search keys before you finally found the actual definition?"
With mandoc's new internal jump targets, this is a problem of the past now
Jasper also sent in a report (http://undeadly.org/cgi?action=article&sid=20150723124332&mode=flat), doing his usual work with Puppet (and specifically "Facter," a tool used by Puppet to gather various bits of system information)
Aside from that and various ports-related work, Jasper worked on adding tame support to some userland tools, fixing some Octeon stuff and introduced something that OpenBSD has oddly lacked until now: an "-i" flag for sed (hooray!)
Antoine Jacoutot gave a report (http://undeadly.org/cgi?action=article&sid=20150722205349&mode=flat) on what he did at the hackathon as well, including improvements to the rcctl tool (for configuring startup services)
It now has an "ls" subcommand with status parsing, allowing you to list running services, stopped services or even ones that failed to start or are supposed to be running (he calls this "the poor man's service monitoring tool")
He also reworked some of the rc.d system to allow smoother operation of multiple instances of the same daemon to run (using tor with different config files as an example)
His list also included updating ports, updating ports documentation, updating the hotplug daemon and laying out some plans for automatic sysmerge for future upgrades
Foundation director Ken Westerback was also there (http://undeadly.org/cgi?action=article&sid=20150722105658&mode=flat), getting some disk-related and laptop work done
He cleaned up and committed the 4k sector softraid code that he'd been working on, as well as fixing some trackpad issues
Stefan Sperling, OpenBSD's token "wireless guy," had a lot to say (http://undeadly.org/cgi?action=article&sid=20150722182236&mode=flat) about the hackathon and what he did there (and even sent in his write-up before he got home)
He taught tcpdump about some new things, including 802.11n metadata beacons (there's a lot more specific detail about this one in the report)
Bringing a bag full of USB wireless devices with him, he set out to get the unsupported ones working, as well as fix some driver bugs in the ones that already did work
One quote from Stefan's report that a lot of people seem to be talking about: "Partway through the hackathon tedu proposed an old diff of his to make our base ls utility display multi-byte characters. This led to a long discussion about how to expand UTF-8 support in base. The conclusion so far indicates that single-byte locales (such as ISO-8859-1 and KOI-8) will be removed from the base OS after the 5.8 release is cut. This simplifies things because the whole system only has to care about a single character encoding. We'll then have a full release cycle to bring UTF-8 support to more base system utilities such as vi, ksh, and mg. To help with this plan, I started organizing a UTF-8-focused hackathon for some time later this year."
Jeremy Evans wrote in (http://undeadly.org/cgi?action=article&sid=20150725180527&mode=flat) to talk about updating lots of ports, moving the ruby ports up to the latest version and also creating perl and ruby wrappers for the new tame subsystem
While he's mainly a ports guy, he got to commit fixes to ports, the base system and even the kernel during the hackathon
Rafael Zalamena, who got commit access at the event, gives his very first report (http://undeadly.org/cgi?action=article&sid=20150725183439&mode=flat) on his networking-related hackathon activities
With Rafael's diffs and help from a couple other developers, OpenBSD now has support for VPLS (https://en.wikipedia.org/wiki/Virtual_Private_LAN_Service)
Jonathan Gray got a lot done (http://undeadly.org/cgi?action=article&sid=20150728184743&mode=flat) in the area of graphics, working on OpenGL and Mesa, updating libdrm and even working with upstream projects to remove some GNU-specific code
As he's become somewhat known for, Jonathan was also busy running three things in the background: clang's fuzzer, cppcheck and AFL (looking for any potential crashes to fix)
Martin Pieuchot gave an write-up (http://undeadly.org/cgi?action=article&sid=20150724183210&mode=flat) on his experience: "I always though that hackathons were the best place to write code, but what's even more important is that they are the best (well actually only) moment where one can discuss and coordinate projects with other developers IRL. And that's what I did."
He laid out some plans for the wireless stack, discussed future plans for PF, made some routing table improvements and did various other bits to the network stack
Unfortunately, most of Martin's secret plans seem to have been left intentionally vague, and will start to take form in the next release cycle
We're still eagerly awaiting a report from one of OpenBSD's newest developers (https://twitter.com/phessler/status/623291827878137856), Alexandr Nedvedicky (the Oracle guy who's working on SMP PF and some other PF fixes)
OpenBSD 5.8's "beta" status was recently reverted, with the message "take that as a hint (https://www.marc.info/?l=openbsd-cvs&m=143766883514831&w=2)," so that may mean more big changes are still to come...
***
FreeBSD quarterly status report (https://www.freebsd.org/news/status/report-2015-04-2015-06.html)
FreeBSD has published their quarterly status report for the months of April to June, citing it to be the largest one so far
It's broken down into a number of sections: team reports, projects, kernel, architectures, userland programs, ports, documentation, Google Summer of Code and miscellaneous others
Starting off with the cluster admin, some machines were moved to the datacenter at New York Internet, email services are now more resilient to failure, the svn mirrors (now just "svn.freebsd.org") are now using GeoGNS with official SSL certs and general redundancy was increased
In the release engineering space, ARM and ARM64 work continues to improve on the Cavium ThunderX, more focus is being put into cloud platforms and the 10.2-RELEASE cycle is reaching its final stages
The core team has been working on phabricator, the fancy review system, and is considering to integrate oauth support soon
Work also continues on bhyve, and more operating systems are slowly gaining support (including the much-rumored Windows Server 2012)
The report also covers recent developments in the Linux emulation layer, and encourages people using 11-CURRENT to help test out the 64bit support
Multipath TCP was also a hot topic, and there's a brief summary of the current status on that patch (it will be available publicly soon)
ZFSguru, a project we haven't talked about a lot, also gets some attention in the report - version 0.3 is set to be completed in early August
PCIe hotplug support is also mentioned, though it's still in the development stages (basic hot-swap functions are working though)
The official binary packages are now built more frequently than before with the help of additional hardware, so AMD64 and i386 users will have fresher ports without the need for compiling
Various other small updates on specific areas of ports (KDE, XFCE, X11...) are also included in the report
Documentation is a strong focus as always, a number of new documentation committers were added and some of the translations have been improved a lot
Many other topics were covered, including foundation updates, conference plans, pkgsrc support in pkgng, ZFS support for UEFI boot and much more
***
The OpenSSH bug that wasn't (http://bsdly.blogspot.com/2015/07/the-openssh-bug-that-wasnt.html)
There's been a lot of discussion (https://www.marc.info/?t=143766048000005&r=1&w=2) about a supposed flaw (https://kingcope.wordpress.com/2015/07/16/openssh-keyboard-interactive-authentication-brute-force-vulnerability-maxauthtries-bypass/) in OpenSSH, allowing attackers to substantially amplify the number of password attempts they can try per session (without leaving any abnormal log traces, even)
There's no actual exploit to speak of; this bug would only help someone get more bruteforce tries in with a fewer number of connections (https://lists.mindrot.org/pipermail/openssh-unix-dev/2015-July/034209.html)
FreeBSD in its default configuration, with PAM (https://en.wikipedia.org/wiki/Pluggable_authentication_module) and ChallengeResponseAuthentication enabled, was the only one vulnerable to the problem - not upstream OpenSSH (https://www.marc.info/?l=openbsd-misc&m=143767296016252&w=2), nor any of the other BSDs, and not even the majority of Linux distros
If you disable all forms of authentication except public keys, like you're supposed to (https://stribika.github.io/2015/01/04/secure-secure-shell.html), then this is also not a big deal for FreeBSD systems
Realistically speaking, it's more of a PAM bug (https://www.marc.info/?l=openbsd-misc&m=143782167322500&w=2) than anything else
OpenSSH added an additional check (https://anongit.mindrot.org/openssh.git/patch/?id=5b64f85bb811246c59ebab) for this type of setup that will be in 7.0, but simply changing your sshd_config is enough to mitigate the issue for now on FreeBSD (or you can run freebsd-update (https://lists.freebsd.org/pipermail/freebsd-security-notifications/2015-July/000248.html))
***
Interview - Sebastian Wiedenroth - wiedi@netbsd.org (mailto:wiedi@netbsd.org) / @wied0r (https://twitter.com/wied0r)
pkgsrc (https://en.wikipedia.org/wiki/Pkgsrc) and pkgsrcCon (http://pkgsrc.org/pkgsrcCon/)
News Roundup
Now served by OpenBSD (https://tribaal.io/this-now-served-by-openbsd.html)
We've mentioned that you can also install OpenBSD on DO droplets, and this blog post is about someone who actually did it
The use case for the author was for a webserver, so he decided to try out the httpd in base
Configuration is ridiculously simple, and the config file in his example provides an HTTPS-only webserver, with plaintext requests automatically redirecting
TLS 1.2 by default, strong ciphers with LibreSSL and HSTS (https://en.wikipedia.org/wiki/HTTP_Strict_Transport_Security) combined give you a pretty secure web server
***
FreeBSD laptop playbooks (https://github.com/sean-/freebsd-laptops)
A new project has started up on Github for configuring FreeBSD on various laptops, unsurprisingly named "freebsd-laptops"
It's based on ansible, and uses the playbook format for automatic set up and configuration
Right now, it's only working on a single Lenovo laptop, but the plan is to add instructions for many more models
Check the Github page for instructions on how to get started, and maybe get involved if you're running FreeBSD on a laptop
***
NetBSD on the NVIDIA Jetson TK1 (https://blog.netbsd.org/tnf/entry/netbsd_on_the_nvidia_jetson)
If you've never heard of the Jetson TK1 (https://developer.nvidia.com/jetson-tk1), we can go ahead and spoil the secret here: NetBSD runs on it
As for the specs, it has a quad-core ARMv7 CPU at 2.3GHz, 2 gigs of RAM, gigabit ethernet, SATA, HDMI and mini-PCIE
This blog post shows which parts of the board are working with NetBSD -current (which seems to be almost everything)
You can even run X11 on it, pretty sweet
***
DragonFly power mangement options (http://lists.dragonflybsd.org/pipermail/users/2015-July/207911.html)
DragonFly developer Sepherosa, who we've had on the show, has been doing some ACPI work over there
In this email, he presents some of DragonFly's different power management options: ACPI P-states, C-states, mwait C-states and some Intel-specific bits as well
He also did some testing with each of them and gave his findings about power saving
If you've been thinking about running DragonFly on a laptop, this would be a good one to read
***
OpenBSD router under FreeBSD bhyve (https://www.quernus.co.uk/2015/07/27/openbsd-as-freebsd-router/)
If one BSD just isn't enough for you, and you've only got one machine, why not run two at once
This article talks about taking a FreeBSD server running bhyve and making a virtualized OpenBSD router with it
If you've been considering switching over your router at home or the office, doing it in a virtual machine is a good way to test the waters before committing to real hardware
The author also includes a little bit of history on how he got into both operating systems
There are lots of mixed opinions about virtualizing core network components, so we'll leave it up to you to do your research
Of course, the next logical step is to put that bhyve host under Xen on NetBSD...
***
Feedback/Questions
Kevin writes in (http://slexy.org/view/s2yPVV5Wyp)
Logan writes in (http://slexy.org/view/s21zcz9rut)
Peter writes in (http://slexy.org/view/s21CRmiPwK)
Randy writes in (http://slexy.org/view/s211zfIXff)
***
| |||
| 24 Feb 2022 | 443: Certified Unix Compliant | 00:46:29 | |
Certifying an OS Unix compliant, 2021 FreeBSD Foundation Impact Report, Netflix, Disney, and other widevine content on FreeBSD, file hashes updated for NetBSD 8.1, Playing with CD-RWs on FreeBSD, Why "process substitution" is a late feature in Unix shells, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
What goes into making an OS to be Unix compliant certified? (https://www.quora.com/What-goes-into-making-an-OS-to-be-Unix-compliant-certified)
2021 FreeBSD Foundation Impact Report (https://freebsdfoundation.org/blog/2021-freebsd-foundation-impact-report/)
News Roundup
Play Netflix, Disney, and other widevine content on FreeBSD (https://danschmid.de/article/play-netflix-disney-and-other-widevine-content-on-freebsd)
Note: two files changed and hashes/signatures updated for NetBSD 8.1 (https://bsdsec.net/articles/note-two-files-changed-and-hashes-signatures-updated-for-netbsd-8-1)
Playing with CD-RWs on FreeBSD (https://rubenerd.com/playing-with-cd-rws-on-freebsd/)
Why "process substitution" is a late feature in Unix shells (https://utcc.utoronto.ca/~cks/space/blog/unix/ProcessSubstitutionWhyLate)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Marty - shell communities (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/443/feedback/Marty%20-%20shell%20communities.md)
Nate - Helping Mike Out (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/443/feedback/Nate%20-%20Helping%20Mike%20Out.md)
Tom - convincing others to switch (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/443/feedback/Tom%20-%20convincing%20others%20to%20switch.md)
***
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 24 Jan 2019 | 282: Open the Rsync | 01:01:20 | |
Project Trident 18.12 released, Spotifyd on NetBSD, OPNsense 18.7.10 is available, Ultra EPYC AMD Powered Sun Ultra 24 Workstation, OpenRsync, LLD porting to NetBSD, and more.
##Headlines ###AsiaBSDCon 2019 Call for Papers
###Project Trident 18.12 Released
###Building Spotifyd on NetBSD
##News Roundup
###Introducing the Ultra EPYC AMD Powered Sun Ultra 24 Workstation
###OpenRsync
###The first report on LLD porting
##Beastie Bits
##Feedback/Questions
| |||
| 20 Jan 2016 | 125: DevSummits, Core and the Baldwin | 02:13:49 | |
This week on the show, we will be talking to FreeBSD developer and former core-team member John Baldwin about a variety of topics, including running a DevSummit, everything you needed or wanted to know. Coming up right now on BSDNow, the place to B...SD.
This episode was brought to you by
Headlines
FreeBSD server retired after almost 19 years (http://www.theregister.co.uk/2016/01/14/server_retired_after_18_years_and_ten_months_beat_that_readers/)
We’ve heard stories about this kind of thing before, that box that often sits under-appreciated, but refuses to die. Well the UK register has picked up on a story of a FreeBSD server finally being retired after almost 19 years of dedicated service.
“In its day, it was a reasonable machine - 200MHz Pentium, 32MB RAM, 4GB SCSI-2 drive,” Ross writes. “And up until recently, it was doing its job fine.” Of late, however the “hard drive finally started throwing errors, it was time to retire it before it gave up the ghost!” The drive's a Seagate, for those of you looking to avoid drives that can't deliver more than 19 years of error-free operations.
This system in particular had been running FreeBSD 2.2.1 over the years. Why not upgrade you ask? Ross has an answer for that:
“It was heavily firewalled and only very specific services were visible to anyone, and most only visible to our directly connected customers,” Ross told Vulture South. “By the time it was probably due for a review, things had moved so far that all the original code was so tightly bound to the operating system itself, that later versions of the OS would have (and ultimately, did) require substantial rework. While it was running and not showing any signs of stress, it was simply expedient to leave sleeping dogs lie.”
All in all, an amazing story of the longevity of a system and its operating system. Do you have a server with a similar or even greater uptime? Let us know so we can try and top this story.
***
Roundup of all the BSDs (https://www.linuxvoice.com/group-test-bsd-distros/)
The magazine LinuxVoice recently did a group test of a variety of “BSD Distros”.
Included in their review were Free/Open/Net/Dragon/Ghost/PC
It starts with a pretty good overview of BSD in general, its starts and the various projects / forks that spawned from it, such as FreeNAS / Junos / Playstation / PFSense / etc
The review starts with a look at OpenBSD, and the consensus reached is that it is good, but does require a bit more manual work to run as a desktop. (Most of the review focuses on desktop usage). It ends up with a solid ⅘ stars though.
Next it moves into GhostBSD, discusses it being a “Live” distro, which can optionally be installed to disk. It loses a few points for lacking a graphical package management utility, and some bugs during the installation, but still earns a respectable ⅗ stars.
Dragonfly gets the next spin and gets praise for its very-up to date video driver support and availability of the HAMMER filesystem. It also lands at ⅗ stars, partly due to the reviewer having to use the command-line for management. (Notice a trend here?)
NetBSD is up next, and gets special mention for being one of the only “distros” that doesn’t do frequent releases. However that doesn’t mean you can’t have updated packages, since the review mentions pkgsrc and pkg as both available to customize your desktop. The reviewer was slightly haunted by having to edit files in /etc by hand to do wireless, but still gives NetBSD a ⅗ overall.
Last up are FreeBSD and PC-BSD, which get a different sort of head-to-head review. FreeBSD goes first, with mention that the text-install is fairly straight-forward and most configuration will require being done by hand. However the reviewer must be getting use to the command-line at this point, because he mentions:
“This might sound cumbersome, but is actually pretty straightforward and at the end produces a finely tuned aerodynamic system that does exactly what you want it to do and nothing else.”
He does mention that FreeBSD is the ultimate DIY system, even to the point of not having the package management tools provided out of box.
PC-BSD ultimately gets a lot of love in this review, again with it being focused on desktop usage this follows. Particularly popular are all the various tools written to make PC-BSD easier to use, such as Life-Preserver, Warden, the graphical installer and more. (slight mistake though, Life-Preserver does not use rsync to backup to FreeNAS, it does ZFS replication)
In the end he rates FreeBSD ⅘ and PC-BSD a whopping 5/5 for this roundup.
While reviews may be subjective to the particular use-case being evaluated for, it is still nice to see BSD getting some press and more interest from the Linux community in general.
***
OpenBSD Laptops (http://www.tedunangst.com/flak/post/openbsd-laptops)
Our buddy Ted Unangst has posted a nice “planning ahead” guide for those thinking of new laptops for 2016 and the upcoming OpenBSD 5.9
He starts by giving us a status update on several of the key driver components that will be in 5.9 release“5.9 will be the first release to support the graphics on Broadwell CPUs. This is anything that looks like i5-5xxx. There are a few minor quirks, but generally it works well. There’s no support for the new Skylake models, however. They’ll probably work with the VESA driver but minus suspend/resume/acceleration (just as 5.8 did with Broadwell).”
He then goes on to mention that the IWM driver works well with most of the revisions (7260, 7265, and 3160) that ship with broadwell based laptops, however the newer skylake series ships with the 8260, which is NOT yet supported.
He then goes on to list some of the more common makes and models to look for, starting with the broadwell based X1 carbons which work really well (Kris gives +++), but make sure its not the newer skylake model just yet.
The macbook gets a mention, but probably should be avoided due to broadcom wifi
The Dell XPS he mentions as a good choice for a powerful (portable) desktops
***
Significant changes from NetBSD 7.0 to 8.0 (https://www.netbsd.org/changes/changes-8.0.html)
Updated to GCC 4.8.5
Imported dhcpcd and replaced rtsol and rtsold
gpt(8) utility gained the ability to resize partitions and disks, as well as change the type of a partition
OpenSSH 7.1 and OpenSSL 1.0.1q
FTP client got support for SNI for https
Imported dtrace from FreeBSD
Add syscall support
Add lockstat support
***
Interview - John Baldwin - jhb@freebsd.org (mailto:jhb@freebsd.org) / @BSDHokie (https://twitter.com/BSDHokie)
FreeBSD Kernel Debugging
News Roundup
Dragonfly Mail Agent spreads to FreeBSD and NetBSD (https://www.dragonflydigest.com/2016/01/18/17508.html)
DMA, the Dragonfly Mail Agent is now available not only in Dragonfly’s dports, but also FreeBSD ports, and NetBSD pkgsrc
“dma is a small Mail Transport Agent (MTA), designed for home and office use. It accepts mails from locally installed Mail User Agents (MUA) and delivers the mails either locally or to a remote destination. Remote delivery includes several features like TLS/SSL support and SMTP authentication. dma is not intended as a replacement for real, big MTAs like sendmail(8) or postfix(1). Consequently, dma does not listen on port 25 for incoming connections.”
There was a project looking at importing DMA into the FreeBSD base system to replace sendmail, I wonder of the port signals that some of the blockers have been fixed
***
ZFS UEFI Support has landed! (https://svnweb.freebsd.org/base?view=revision&revision=294068)
Originally started by Eric McCorkle
Picked up by Steven Hartland
Including modularizing the existing UFS boot code, and adding ZFS boot code
General improvements to the EFI loader including using more of libstand instead of containing its own implementations of many common functions
Thanks to work by Toomas Soome, there is now a Beastie Menu as part of the EFI loader, similar to the regular loader
As soon as this was committed, I added a few lines to it to connect the ZFS BE Menu to it, thanks to all of the above, without whom my work wouldn’t be usable
It should be relatively easy to hook my GELI boot stuff in as a module, and possibly just stack the UFS and ZFS modules on top of it
I might try to redesign the non-EFI boot code to use a similar design instead of what I have now
***
How three BSD OSes compare to ten Linux Distros (http://www.phoronix.com/scan.php?page=article&item=3bsd-10linux)
After benchmarking 10 of the latest Linux distros, Phoronix took to benchmarking 3 of the big BSDs
DragonFlyBSD 4.4.1 - The latest DragonFly release with GCC 5.2.1 and the HAMMER file-system.
OpenBSD 5.8 - OpenBSD 5.8 with GCC 4.2.1 as the default compiler and FFS file-system.
PC-BSD 10.2 - Derived off FreeBSD 10.2, the defaults were the Clang 3.4.1 compiler and ZFS file-system.
In the SQLite test, PCBSD+ZFS won out over all of the Linux distros, including those that were also using ZFS
In the first compile benchmark, PCBSD came second only to Intel’s Linux distro, Clear Linux. OpenBSD can last, although it is not clear if the benchmark was just comparing the system compiler, which would be unfair to OpenBSD
In Disk transaction performance, against ZFS won the day, with PCBSD edging out the Linux distros. OpenBSD’s older ffs was hurt by the lack of soft updates, and DragonFly’s Hammer did not perform well. Although in an fsync() heavy test, safety is more important that speed
As with all benchmarks, these obviously need to be taken with a grain of salt
In some of them you can clearly see that the ‘winner’ has a much higher standard error, suggesting that the numbers are quite variable
***
OPNSense 15.7.24 Released (https://opnsense.org/opnsense-15-7-24-released/)
We are just barely into the new year and OPNSense has dropped a new release on us to play with.
This new version, 15.7.24 brings a bunch of notable changes, which includes improvements to the firewall UI and a plugin management section of the firmware page. Additionally better signature verification using PKG’s internal verification mechanisms was added for kernel and world updates.
The announcement contains the full rundown of changes, including the suricata, openvpn and ntp got package bumps as well.
***
Beastie Bits
A FreeBSD 10 Desktop How-to (https://cooltrainer.org/a-freebsd-desktop-howto/) (A bit old, but still one of the most complete walkthroughs of a desktop FreeBSD setup from scratch)
BSD and Scale 14 (http://fossforce.com/2016/01/bsd-ready-scale-14x/)
Xen support enabled in OpenBSD -current (http://undeadly.org/cgi?action=article&sid=20160114113445&mode=expanded)
Feedback/Questions
Matt - Zil Sizes (http://slexy.org/view/s20a0mLaAv)
Drin - IPSEC (http://slexy.org/view/s21qpiTF8h)
John - ZFS + UEFI (http://slexy.org/view/s2HCq0r0aD)
Jake - ZFS Cluster SAN (http://slexy.org/view/s2VORfyqlS)
Phillip - Media Server (http://slexy.org/view/s20ycRhUkM)
***
| |||
| 07 Mar 2024 | 549: htop Tetris | 00:56:46 | |
FreeBSD Foundation Statement on the European Union Cyber Resiliency Act, DragonFly BSD on a Thinkpad T480s, How FreeBSD
Employs Ampere Arm64 Servers in the Data Center, FreeBSD Yubikey authentication, that time I almost added Tetris to htop, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
FreeBSD Foundation Statement on the European Union Cyber Resiliency Act (https://freebsdfoundation.org/blog/freebsd-foundation-statement-on-the-european-union-cyber-resiliency-act/?utm_source=bsdweekly)
DragonFly BSD on a Thinkpad T480s (https://git.sr.ht/~tomh/dragonflybsd-on-a-laptop/tree/master/item/README.md)
News Roundup
Ampere in the Wild: How FreeBSD Employs Ampere Arm64 Servers in the Data Center (https://amperecomputing.com/blogs/ampere-in-the-wild)
FreeBSD Yubikey authentication (https://gist.github.com/daemonhorn/bdd77a7bc0ff5842e5a31d999b96e1f1)
That time I almost added Tetris to htop (https://hisham.hm/2024/02/12/that-time-i-almost-added-tetris-to-htop/)
Beastie Bits
Mail Software Projects for You (https://mwl.io/archives/23419)
At long last: the MWL Title Index (https://mwl.io/archives/23401)
FreeBSD on a RPi (https://linux.slashdot.org/story/24/01/07/0327229/how-does-freebsd-compare-to-linux-on-a-raspberry-pi)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 21 Jun 2017 | 199: Read the source, KARL | 01:22:11 | |
FreeBSD 11.1-Beta1 is out, we discuss Kernel address randomized link (KARL), and explore the benefits of daily OpenBSD source code reading
This episode was brought to you by
Headlines
FreeBSD 11.1-Beta1 now available (https://lists.freebsd.org/pipermail/freebsd-stable/2017-June/087242.html)
Glen Barber, of the FreeBSD release engineering team has announced that FreeBSD 11.1-Beta1 is now available for the following architectures:
11.1-BETA1 amd64 GENERIC
11.1-BETA1 i386 GENERIC
11.1-BETA1 powerpc GENERIC
11.1-BETA1 powerpc64 GENERIC64
11.1-BETA1 sparc64 GENERIC
11.1-BETA1 armv6 BANANAPI
11.1-BETA1 armv6 BEAGLEBONE
11.1-BETA1 armv6 CUBIEBOARD
11.1-BETA1 armv6 CUBIEBOARD2
11.1-BETA1 armv6 CUBOX-HUMMINGBOARD
11.1-BETA1 armv6 GUMSTIX
11.1-BETA1 armv6 RPI-B
11.1-BETA1 armv6 RPI2
11.1-BETA1 armv6 PANDABOARD
11.1-BETA1 armv6 WANDBOARD
11.1-BETA1 aarch64 GENERIC
Note regarding arm/armv6 images: For convenience for those without console access to the system, a freebsd user with a password of freebsd is available by default for ssh(1) access. Additionally, the root user password is set to root. It is strongly recommended to change the password for both users after gaining access to the system.
The full schedule (https://www.freebsd.org/releases/11.1R/schedule.html) for 11.1-RELEASE is here, the final release is expected at the end of July
It was also announced there will be a 10.4-RELEASE scheduled for October (https://www.freebsd.org/releases/10.4R/schedule.html)
***
KARL – kernel address randomized link (https://marc.info/?l=openbsd-tech&m=149732026405941&w=2)
Over the last three weeks I've been working on a new randomization feature which will protect the kernel.
The situation today is that many people install a kernel binary from OpenBSD, and then run that same kernel binary for 6 months or more. We have substantial randomization for the memory allocations made by the kernel, and for userland also of course.
Previously, the kernel assembly language bootstrap/runtime locore.S was compiled and linked with all the other .c files of the kernel in a deterministic fashion. locore.o was always first, then the .c files order specified by our config(8) utility and some helper files.
In the new world order, locore is split into two files: One chunk is bootstrap, that is left at the beginning. The assembly language runtime and all other files are linked in random fashion. There are some other pieces to try to improve the randomness of the layout.
As a result, every new kernel is unique. The relative offsets between functions and data are unique.
It still loads at the same location in KVA. This is not kernel ASLR! ASLR is a concept where the base address of a module is biased to a random location, for position-independent execution. In this case, the module itself is perturbed but it lands at the same location, and
does not need to use position-independent execution modes.
LLDB: Sanitizing the debugger's runtime (https://blog.netbsd.org/tnf/entry/lldb_sanitizing_the_debugger_s)
The good
Besides the greater enhancements this month I performed a cleanup in the ATF ptrace(2) tests again. Additionally I have managed to unbreak the LLDB Debug build and to eliminate compiler warnings in the NetBSD Native Process Plugin. It is worth noting that LLVM can run tests on NetBSD again, the patch in gtest/LLVM has been installed by Joerg Sonnenberg and a more generic one has been submitted to the upstream googletest repository. There was also an improvement in ftruncate(2) on the LLVM side (authored by Joerg).
Since LLD (the LLVM linker) is advancing rapidly, it improved support for NetBSD and it can link a functional executable on NetBSD. I submitted a patch to stop crashing it on startup anymore. It was nearly used for linking LLDB/NetBSD and it spotted a real linking error... however there are further issues that need to be addressed in the future. Currently LLD is not part of the mainline LLDB tasks - it's part of improving the work environment. This linker should reduce the linking time - compared to GNU linkers - of LLDB by a factor of 3x-10x and save precious developer time. As of now LLDB linking can take minutes on a modern amd64 machine designed for performance.
Kernel correctness
I have researched (in pkgsrc-wip) initial support for multiple threads in the NetBSD Native Process Plugin. This code revealed - when running the LLDB regression test-suite - new kernel bugs. This unfortunately affects the usability of a debugger in a multithread environment in general and explains why GDB was never doing its job properly in such circumstances. One of the first errors was asserting kernel panic with PT*STEP, when a debuggee has more than a single thread. I have narrowed it down to lock primitives misuse in the doptrace() kernel code. The fix has been committed.
The bad
Unfortunately this is not the full story and there is further mandatory work.
LLDB acceleration
The EV_SET() bug broke upstream LLDB over a month ago, and during this period the debugger was significantly accelerated and parallelized. It is difficult to declare it definitely, but it might be the reason why the tracer's runtime broke due to threading desynchronization. LLDB behaves differently when run standalone, under ktruss(1) and under gdb(1) - the shared bug is that it always fails in one way or another, which isn't trivial to debug.
The ugly
There are also unpleasant issues at the core of the Operating System.
Kernel troubles
Another bug with single-step functions that affects another aspect of correctness - this time with reliable execution of a program - is that processes die in non-deterministic ways when single-stepped. My current impression is that there is no appropriate translation between process and thread (LWP) states under a debugger. These issues are sibling problems to unreliable PTRESUME and PTSUSPEND.
In order to be able to appropriately address this, I have diligently studied this month the Solaris Internals book to get a better image of the design of the NetBSD kernel multiprocessing, which was modeled after this commercial UNIX.
Plan for the next milestone
The current troubles can be summarized as data races in the kernel and at the same time in LLDB. I have decided to port the LLVM sanitizers, as I require the Thread Sanitizer (tsan). Temporarily I have removed the code for tracing processes with multiple threads to hide the known kernel bugs and focus on the LLDB races.
Unfortunately LLDB is not easily bisectable (build time of the LLVM+Clang+LLDB stack, number of revisions), therefore the debugging has to be performed on the most recent code from upstream trunk.
d2K17 Hackathon Reports
d2k17 Hackathon Report: Ken Westerback on XSNOCCB removal and dhclient link detection (http://undeadly.org/cgi?action=article&sid=20170605225415)
d2k17 Hackathon Report: Antoine Jacoutot on rc.d, syspatch, and more (http://undeadly.org/cgi?action=article&sid=20170608074033)
d2k17 Hackathon Report: Florian Obser on slaacd(8) (http://undeadly.org/cgi?action=article&sid=20170609013548)
d2k17 Hackathon Report: Stefan Sperling on USB audio, WiFi Progress (http://undeadly.org/cgi?action=article&sid=20170602014048)
News Roundup
Multi-tenant router or firewall with FreeBSD (https://bsdrp.net/documentation/examples/multi-tenant_router_and_firewall)
Setting-up a virtual lab
Downloading BSD Router Project images
Download BSDRP serial image (prevent to have to use an X display) on Sourceforge.
Download Lab scripts
More information on these BSDRP lab scripts available on How to build a BSDRP router lab (https://bsdrp.net/documentation/examples/how_to_build_a_bsdrp_router_lab).
Start the lab with full-meshed 5 routers and one shared LAN, on this example using bhyve lab script on FreeBSD:
[root@FreeBSD]~# tools/BSDRP-lab-bhyve.sh -i BSDRP-1.71-full-amd64-serial.img.xz -n 5 -l 1
Configuration
Router 4 (R4) hosts the 3 routers/firewalls for each 3 customers.
Router 1 (R1) belongs to customer 1, router 2 (R2) to customer 2 and router 3 (R3) to customer 3.
Router 5 (R5) simulates a simple Internet host
Using pf firewall in place of ipfw
pf need a little more configuration because by default /dev/pf is hidden from jail. Then, on the host we need to:
In place of loading the ipfw/ipfw-nat modules we need to load the pf module (but still disabling pf on our host for this example)
Modify default devd rules for allowing jails to see /dev/pf (if you want to use tcpdump inside your jail, you should use bpf device too)
Replacing nojail tag by nojailvnet tag into /etc/rc.d/pf (already done into BSDRP (https://github.com/ocochard/BSDRP/blob/master/BSDRP/patches/freebsd.pf.rc.jail.patch))
Under the hood: jails-on-nanobsd
BSDRP's tenant shell script (https://github.com/ocochard/BSDRP/blob/master/BSDRP/Files/usr/local/sbin/tenant) creates jail configuration compliant with a host running nanobsd. Then these jails need to be configured for a nanobsd:
Being nullfs based for being hosted on a read-only root filesystem
Have their /etc and /var into tmpfs disks (then we need to populate these directory before each start)
Configuration changes need to be saved with nanobsd configuration tools, like “config save” on BSDRP
And on the host:
autosave daemon (https://github.com/ocochard/BSDRP/blob/master/BSDRP/Files/usr/local/sbin/autosave) need to be enabled: Each time a customer will issue a “config save” inside a jail, his configuration diffs will be save into host's /etc/jails/. And this directory is a RAM disk too, then we need to automatically save hosts configuration on changes.
***
OpenBSD Daily Source Reading (https://blog.tintagel.pl/2017/06/09/openbsd-daily.html)
Adam Wołk writes:
I made a new year's resolution to read at least one C source file from OpenBSD daily. The goal was to both get better at C and to contribute more to the base system and userland development. I have to admit that initially I wasn’t consistent with it at all. In the first quarter of the year I read the code of a few small base utilities and nothing else. Still, every bit counts and it’s never too late to get better.
Around the end of May, I really started reading code daily - no days skipped. It usually takes anywhere between ten minutes (for small base utils) and one and a half hour (for targeted reads). I’m pretty happy with the results so far. Exploring the system on a daily basis, looking up things in the code that I don’t understand and digging as deep as possible made me learn a lot more both about C and the system than I initially expected.
There’s also one more side effect of reading code daily - diffs. It’s easy to spot inconsistencies, outdated code or an incorrect man page. This results in opportunities for contributing to the project. With time it also becomes less opportunitstic and more goal oriented. You might start with a https://marc.info/?l=openbsd-tech&m=149591302814638&w=2 (drive by diff to kill) optional compilation of an old compatibility option in chown that has been compiled in by default since 1995.
Soon the contributions become more targeted, for example using a new API for encrypting passwords in the htpasswd utility after reading the code of the utility and the code for htpasswd handling in httpd. Similarly it can take you from discussing a doas feature idea with a friend to implementing it after reading the code.
I was having a lot of fun reading code daily and started to recommend it to people in general discussions. There was one particular twitter thread that ended up starting something new.
This is still a new thing and the format is not yet solidified. Generally I make a lot of notes reading code, instead of slapping them inside a local file I drop the notes on the IRC channel as I go. Everyone on the channel is encouraged to do the same or share his notes in any way he/she seems feasable.
Check out the logs from the IRC discussions.
Start reading code from other BSD projects and see whether you can replicate their results!
***
Become FreeBSD User: Find Useful Tools (https://bsdmag.org/become-freebsd-user-find-useful-tools/)
BSD Mag has the following article by David Carlier:
If you’re usually programming on Linux and you consider a potential switch to FreeBSD, this article will give you an overview of the possibilities.
How to Install the Dependencies
FreeBSD comes with either applications from binary packages or compiled from sources (ports). They are arranged according to software types (programming languages mainly in lang (or java specifically for Java), libraries in devel, web servers in www …) and the main tool for modern FreeBSD versions is pkg, similar to Debian apt tools suite. Hence, most of the time if you are looking for a specific application/library, simply
pkg search | |||
| 22 Mar 2017 | 186: The Fast And the Firewall: Tokyo Drift | 02:54:07 | |
This week on BSDNow, reports from AsiaBSDcon, TrueOS and FreeBSD news, Optimizing IllumOS Kernel, your questions and more.
This episode was brought to you by
Headlines
AsiaBSDcon Reports and Reviews ()
AsiaBSDcon schedule (https://2017.asiabsdcon.org/program.html.en)
Schedule and slides from the 4th bhyvecon (http://bhyvecon.org/)
Michael Dexter’s trip report on the iXsystems blog (https://www.ixsystems.com/blog/ixsystems-attends-asiabsdcon-2017)
NetBSD AsiaBSDcon booth report (http://mail-index.netbsd.org/netbsd-advocacy/2017/03/13/msg000729.html)
***
TrueOS Community Guidelines are here! (https://www.trueos.org/blog/trueos-community-guidelines/)
TrueOS has published its new Community Guidelines
The TrueOS Project has existed for over ten years. Until now, there was no formally defined process for interested individuals in the TrueOS community to earn contributor status as an active committer to this long-standing project. The current core TrueOS developers (Kris Moore, Ken Moore, and Joe Maloney) want to provide the community more opportunities to directly impact the TrueOS Project, and wish to formalize the process for interested people to gain full commit access to the TrueOS repositories.
These describe what is expected of community members and committers
They also describe the process of getting commit access to the TrueOS repo:
Previously, Kris directly handed out commit bits. Now, the Core developers have provided a small list of requirements for gaining a TrueOS commit bit:
Create five or more pull requests in a TrueOS Project repository within a single six month period.
Stay active in the TrueOS community through at least one of the available community channels (Gitter, Discourse, IRC, etc.).
Request commit access from the core developers via core@trueos.org OR Core developers contact you concerning commit access.
Pull requests can be any contribution to the project, from minor documentation tweaks to creating full utilities.
At the end of every month, the core developers review the commit logs, removing elements that break the Project or deviate too far from its intended purpose. Additionally, outstanding pull requests with no active dissension are immediately merged, if possible. For example, a user submits a pull request which adds a little-used OpenRC script. No one from the community comments on the request or otherwise argues against its inclusion, resulting in an automatic merge at the end of the month. In this manner, solid contributions are routinely added to the project and never left in a state of “limbo”.
The page also describes the perks of being a TrueOS committer:
Contributors to the TrueOS Project enjoy a number of benefits, including:
A personal TrueOS email alias: | |||
| 02 Sep 2015 | 105: Virginia BSD Assembly | 01:06:09 | |
It's already our two-year anniversary! This time on the show, we'll be chatting with Scott Courtney, vice president of infrastructure engineering at Verisign, about this year's vBSDCon. What's it have to offer in an already-crowded BSD conference space? We'll find out.
This episode was brought to you by
Headlines
OpenBSD hypervisor coming soon (https://www.marc.info/?l=openbsd-tech&m=144104398132541&w=2)
Our buddy Mike Larkin never rests, and he posted some very tight-lipped console output (http://pastebin.com/raw.php?i=F2Qbgdde) on Twitter recently
From what little he revealed at the time (https://twitter.com/mlarkin2012/status/638265767864070144), it appeared to be a new hypervisor (https://en.wikipedia.org/wiki/Hypervisor) (that is, X86 hardware virtualization) running on OpenBSD -current, tentatively titled "vmm"
Later on, he provided a much longer explanation on the mailing list, detailing a bit about what the overall plan for the code is
Originally started around the time of the Australia hackathon, the work has since picked up more steam, and has gotten a funding boost from the OpenBSD foundation
One thing to note: this isn't just a port of something like Xen or Bhyve; it's all-new code, and Mike explains why he chose to go that route
He also answered some basic questions about the requirements, when it'll be available, what OSes it can run, what's left to do, how to get involved and so on
***
Why FreeBSD should not adopt launchd (http://blog.darknedgy.net/technology/2015/08/26/0/)
Last week (http://www.bsdnow.tv/episodes/2015_08_26-beverly_hills_25519) we mentioned a talk Jordan Hubbard gave about integrating various parts of Mac OS X into FreeBSD
One of the changes, perhaps the most controversial item on the list, was the adoption of launchd to replace the init system (replacing init systems seems to cause backlash, we've learned)
In this article, the author talks about why he thinks this is a bad idea
He doesn't oppose the integration into FreeBSD-derived projects, like FreeNAS and PC-BSD, only vanilla FreeBSD itself - this is also explained in more detail
The post includes both high-level descriptions and low-level technical details, and provides an interesting outlook on the situation and possibilities
Reddit had quite a bit (https://www.reddit.com/r/BSD/comments/3ilhpk) to say (https://www.reddit.com/r/freebsd/comments/3ilj4i) about this one, some in agreement and some not
***
DragonFly graphics improvements (http://lists.dragonflybsd.org/pipermail/commits/2015-August/458108.html)
The DragonFlyBSD guys are at it again, merging newer support and fixes into their i915 (Intel) graphics stack
This latest update brings them in sync with Linux 3.17, and includes Haswell fixes, DisplayPort fixes, improvements for Broadwell and even Cherryview GPUs
You should also see some power management improvements, longer battery life and various other bug fixes
If you're running DragonFly, especially on a laptop, you'll want to get this stuff on your machine quick - big improvements all around
***
OpenBSD tames the userland (https://www.marc.info/?l=openbsd-tech&m=144070638327053&w=2)
Last week we mentioned OpenBSD's tame framework getting support for file whitelists, and said that the userland integration was next - well, now here we are
Theo posted a mega diff of nearly 100 smaller diffs, adding tame support to many areas of the userland tools
It's still a work-in-progress version; there's still more to be added (including the file path whitelist stuff)
Some classic utilities are even being reworked to make taming them easier - the "w" command (https://www.marc.info/?l=openbsd-cvs&m=144103945031253&w=2), for example
The diff provides some good insight on exactly how to restrict different types of utilities, as well as how easy it is to actually do so (and en masse)
More discussion can be found on HN (https://news.ycombinator.com/item?id=10135901), as one might expect
If you're a software developer, and especially if your software is in ports already, consider adding some more fine-grained tame support in your next release
***
Interview - Scott Courtney - vbsdcon@verisign.com (mailto:vbsdcon@verisign.com) / @verisign (https://twitter.com/verisign)
vBSDCon (http://vbsdcon.com/) 2015
News Roundup
OPNsense, beyond the fork (https://opnsense.org/opnsense-beyond-the-fork)
We first heard about (http://www.bsdnow.tv/episodes/2015_01_14-common_sense_approach) OPNsense back in January, and they've since released nearly 40 versions, spanning over 5,000 commits
This is their first big status update, covering some of the things that've happened since the project was born
There's been a lot of community growth and participation, mass bug fixing, new features added, experimental builds with ASLR and much more - the report touches on a little of everything
***
LibreSSL nukes SSLv3 (http://undeadly.org/cgi?action=article&sid=20150827112006)
With their latest release, LibreSSL began to turn off SSLv3 (http://disablessl3.com) support, starting with the "openssl" command
At the time, SSLv3 wasn't disabled entirely because of some things in the OpenBSD ports tree requiring it (apache being one odd example)
They've now flipped the switch, and the process of complete removal has started
From the Undeadly summary, "This is an important step for the security of the LibreSSL library and, by extension, the ports tree. It does, however, require lots of testing of the resulting packages, as some of the fallout may be at runtime (so not detected during the build). That is part of why this is committed at this point during the release cycle: it gives the community more time to test packages and report issues so that these can be fixed. When these fixes are then pushed upstream, the entire software ecosystem will benefit. In short: you know what to do!"
With this change and a few more to follow shortly, LibreSSL won't actually support SSL anymore - time to rename it "LibreTLS"
***
FreeBSD MPTCP updated (http://caia.swin.edu.au/urp/newtcp/mptcp/tools/v05/mptcp-readme-v0.5.txt)
For anyone unaware, Multipath TCP (https://en.wikipedia.org/wiki/Multipath_TCP) is "an ongoing effort of the Internet Engineering Task Force's (IETF) Multipath TCP working group, that aims at allowing a Transmission Control Protocol (TCP) connection to use multiple paths to maximize resource usage and increase redundancy."
There's been work out of an Australian university to add support for it to the FreeBSD kernel, and the patchset was recently updated
Including in this latest version is an overview of the protocol, how to get it compiled in, current features and limitations and some info about the routing requirements
Some big performance gains can be had with MPTCP, but only if both the client and server systems support it - getting it into the FreeBSD kernel would be a good start
***
UEFI and GPT in OpenBSD (https://www.marc.info/?l=openbsd-cvs&m=144092912907778&w=2)
There hasn't been much fanfare about it yet, but some initial UEFI and GPT-related commits have been creeping into OpenBSD recently
Some support (https://github.com/yasuoka/openbsd-uefi) for UEFI booting has landed in the kernel, and more bits are being slowly enabled after review
This comes along with a number (https://www.marc.info/?l=openbsd-cvs&m=143732984925140&w=2) of (https://www.marc.info/?l=openbsd-cvs&m=144088136200753&w=2) other (https://www.marc.info/?l=openbsd-cvs&m=144046793225230&w=2) commits (https://www.marc.info/?l=openbsd-cvs&m=144045760723039&w=2) related to GPT, much of which is being refactored and slowly reintroduced
Currently, you have to do some disklabel wizardry to bypass the MBR limit and access more than 2TB of space on a single drive, but it should "just work" with GPT (once everything's in)
The UEFI bootloader support has been committed (https://www.marc.info/?l=openbsd-cvs&m=144115942223734&w=2), so stay tuned for more updates (http://undeadly.org/cgi?action=article&sid=20150902074526&mode=flat) as further (https://twitter.com/kotatsu_mi/status/638909417761562624) progress (https://twitter.com/yojiro/status/638189353601097728) is made
***
Feedback/Questions
John writes in (http://slexy.org/view/s2sIWfb3Qh)
Mason writes in (http://slexy.org/view/s2Ybrx00KI)
Earl writes in (http://slexy.org/view/s20FpmR7ZW)
***
| |||
| 05 Oct 2023 | 527: Reports are in | 01:00:43 | |
Unlocking Infrastructure Sovereignty, first meeting of the FreeBSD Enterprise Working Group, HardenedBSD August 2023 Status Report, GhostBSD August 2023 donation report, MidnightBSD 3.1 Released, OpenBSD Webzine ISSUE #14, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Unlocking Infrastructure Sovereignty: Harnessing the Power of Open Source Solutions for Business Flexibility and Cost-Effectiveness (https://klarasystems.com/articles/unlocking-infrastructure-sovereignty-harnessing-the-power-of-open-source-solutions/)
Recap of first meeting of the FreeBSD Enterprise Working Group (https://freebsdfoundation.org/blog/recap-of-first-meeting-of-the-freebsd-enterprise-working-group/)
News Roundup
HardenedBSD August 2023 Status Report (https://hardenedbsd.org/article/shawn-webb/2023-09-01/hardenedbsd-august-2023-status-report)
• [HardenedBSD 14-STABLE Now Available](https://hardenedbsd.org/article/shawn-webb/2023-09-11/hardenedbsd-14-stable-now-available)
August 2023 donation report (http://ghostbsd.org/news/August_2023_donation_report)
• [Late on the announcement but... GhostBSD 23.06.01 ISO is now available](http://ghostbsd.org/23.06.01_iso_is_now_available)
MidnightBSD 3.1 Released (https://www.phoronix.com/news/MidnightBSD-3.1)
OpenBSD Webzine ISSUE #14 is out (https://webzine.puffy.cafe/issue-14.html)
Beastie Bits
• [ZFS for Dummies](https://ikrima.dev/dev-notes/homelab/zfs-for-dummies/)
• [The Switch runs FreeBSD](https://www.reddit.com/r/NintendoSwitch/comments/5xbe5a/the_switch_runs_freebsd_making_it_nintendos_first/)
• [KDE on OpenBSD](https://marc.info/?l=openbsd-ports&m=169391479324962)
• [(Kubernetes v1.28.0) for illumos, FreeBSD and OpenBSD](https://medium.com/@norlin.t/by-the-way-planternetes-kubernetes-v1-28-0-for-illumos-freebsd-and-openbsd-5d57026d6a25)
• [Video: C Programming on System 6 - VCF Midwest, Wi-Fi DA](https://jcs.org/2023/09/20/vcfmw)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 18 May 2016 | 142: Diving for BSD Perls | 01:36:51 | |
This week on the show, we have all the latest news and stories! Plus an interview with BSD developer Alfred Perlstein, that you
This episode was brought to you by
Headlines
The May issus of BSDMag is now out (https://bsdmag.org/download/reusing_openbsd/)
GhostBSD
Reusing OpenBSD's arc4random in multi-threaded user space programs
Securing VPN's with GRE / Strongswan
Installing XFCE 4.12 on NetBSD 7
Interview with Fernando Rodriguez, the co-founder of KeepCoding
***
A rundown of the FPTW^XEXT.1 security reqiurement for General Purpose Operating Systems by the NSA (http://blog.acumensecurity.net/fpt_wx_ext-1-a-rundown/)
NIST/NSA Validation Scheme Report (https://www.commoncriteriaportal.org/files/ppfiles/pp_os_v4.1-vr.pdf)
The SFR or Security Functional Requirement requires that; "The OS shall prevent allocation of any memory region with both write and execute permissions except for [assignment: list of exceptions]."
While nearly all operating systems currently support the use of the NX bit, or the equivalent on processors such as SPARC and ARM, and will correctly mark the stack as non-executable, the fact remains that this in and of itself is deemed insufficient by NIST and NSA.
OpenBSD 5.8, FreeBSD, Solaris, RHEL, and most other Linux distro have failed.
HardenedBSD passes all three tests out of the box.
NetBSD will do so with a single sysctl tweak. Since they are using the PaX model, anything else using PaX, such as a grsecurity-enabled Linux distribution pass these assurance activities as well.
OpenBSD 5.9 does not allow memory mapping due to W^X being enforced by the kernel, however the kernel will panic if there are any attempts to create such mappings.
***
DistroWatch reviews new features in FreeBSD 10.3 (https://distrowatch.com/weekly.php?issue=20160516#freebsd)
DistroWatch did a review of FreeBSD 10.3
They ran into a few problems, but hopefully those can be fixed
An issue with beadm setting the canmount property incorrectly causing the ZFS BE menu to not work as expected should be resolved in the next version, thanks to a patch from kmoore
The limitations of the Linux 64 support are what they are, CentOS 6 is still fairly popular with enterprise software, but hopefully some folks are interested in working on bringing the syscall emulation forward
In a third issue, the reviewer seemed to have issues SSHing from inside the jail. This likely has to do with how they got a console in the jail. I remember having problems with this in the past, something about a secure console.
***
BSD Unix: Power to the people, from the code (https://www.salon.com/2000/05/16/chapter_2_part_one/)
Salon.com has a very long article, chronicling much of the history behind BSD UNIX.
It starts with detailing the humble origins of BSD, starting with Bill Joy in the mid-70’s, and then goes through details on how it rapidly grew, and the influence that the University of Berkeley had on open-source.
“But too much focus on Joy, a favorite target for business magazine hagiography, obscures the larger picture. Berkeley’s most important contribution was not software; it was the way Berkeley created software. At Berkeley, a small core group — never more than four people at any one time — coordinated the contributions of an ever-growing network of far-flung, mostly volunteer programmers into progressive releases of steadily improving software. In so doing, they codified a template for what is now referred to as the “open-source software development methodology.” Put more simply, the Berkeley hackers set up a system for creating free software.”
The article goes on to talk about some of the back and forth between Linux and BSD, and why Linux has captured more of the market in recent years, but BSD is far from throwing in the towel.
“BSD patriots argue that the battle is far from over, that BSD is technically superior and will therefore win in the end. That’s for the future to determine. What’s indisputable is BSD’s contribution in the past. Even if, by 1975, Berkeley’s Free Speech Movement was a relic belonging to a fast-fading generation, on the fourth floor of Evans Hall, where Joy shared an office, the free-software movement was just beginning.”
An excellent article (If a bit long), but well worth your time to understand the origins of what we consider modern day BSD, and how the University of Berkley helped shape it.
***
iXsystems (http://ixsystems.com)
#ServerEnvy: It's over 10,000 Terabytes! (https://www.ixsystems.com/blog/serverenvy-10000-terabytes/)
***
Interview - Alfred Perlstein - alfred@freebsd.org (mailto:alfred@freebsd.org) / @splbio (https://twitter.com/splbio)
Using BSD for projects
***
News Roundup
.NET framework ported to NetBSD (https://github.com/dotnet/coreclr/pull/4504/files)
This pull request adds basic support for the .NET framework on NetBSD 7.x amd64
It includes documentation on how to get the .NET framework installed
It uses pkgsrc to bootstrap the required tools
pkgsrc-wip is used to get the actual .NET framework, as porting is still in progress
The .NET Core-CLR is now available for: FreeBSD, Linux, NetBSD, and OS X
***
OpenBSD SROP mitigation – call for testing (https://marc.info/?l=openbsd-tech&m=146281531025185&w=2)
A new technique for exploiting flaws in applications and operating systems has been developed, called SROP
“we describe Sigreturn Oriented Programming (SROP), a novel technique for exploits and backdoors in UNIX-like systems. Like return-oriented programming (ROP), sigreturn oriented programming constructs what is known as a ‘weird machine’ that can be programmed by attackers to change the behavior of a process. To program the machine, attackers set up fake signal frames and initiate returns from signals that the kernel never really delivered. This is possible, because UNIX stores signal frames on the process’ stack.”
“Sigreturn oriented programming is interesting for attackers, OS developers and academics. For attackers, the technique is very versatile, with pre-conditions that are different from those of existing exploitation techniques like ROP. Moreover, unlike ROP, sigreturn oriented programming programs are portable. For OS developers, the technique presents a problem that has been present in one of the two main operating system families from its inception, while the fixes (which we also present) are non-trivial. From a more academic viewpoint, it is also interesting because we show that sigreturn oriented programming is Turing complete.”
Paper describing SROP (http://www.cs.vu.nl/~herbertb/papers/srop_sp14.pdf)
OpenBSD has developed a mitigation against SROP
“Utilizing a trick from kbind(2), the kernel now only accepts signal returns from the PC address of the sigreturn(2) syscall in the signal trampoline. Since the signal trampoline page is randomized placed per process, it is only known by directly returning from a signal handler.”
“As well, the sigcontext provided to sigreturn(2) now contains a magic cookie constructed from a per-process cookie XOR'd against the address of the signal context.”
This is just a draft of the patch, not yet considered production quality
***
Running Tor in a NetBSD rump unikernel (https://github.com/supradix/rumprun-packages/tree/33d9cc3a65a39e32b4bc8034c151a5d7e0b89f66/tor)
We’ve talked about “rump” kernels before, and also Tor pretty frequently, but this new github project combines the two!
Specifically, this set of Makefile and scripts will prep a system to run Tor via the Unikernel through Qemu.
The script mainly describes how to do the initial setup on Linux, using iptables, but could easily be adapted to a BSD if somebody wants to do so. (Send them a pull request with the instructions!)
All in all, this is a fascinating way to run a Tor node or relay, in the most minimal operating environment possible.
***
An update on SSH protocol 1 ("we're most of the way towards fully deprecating SSH protocol 1" (http://lists.mindrot.org/pipermail/openssh-unix-dev/2016-May/035069.html)
Damien Miller has given us an update on the status of the “SSH protocol 1”, and the current plans to deprecate it in an upcoming version of openssh.
“We've had this old protocol in various stages of deprecation for almost 10 years and it has been compile-time disabled for about a year.
Downstream vendors, to their credit, have included this change in recent OS releases by shipping OpenSSH packages that disable protocol 1 by default and/or offering separate, non-default packages to enable it.
This seems to have proceeded far more smoothly than even my most optimistic hopes, so this gives us greater confidence that we can complete the removal of protocol 1 soon. We want to do this partly to hasten the demise of this cryptographic trainwreck, but also because doing so removes a lot of legacy code from OpenSSH that inflates our attack surface. Having it gone will make our jobs quite a bit easier as we maintain and refactor.”
The current time-line looks like removing server-size protocol 1 support this August after OpenSSH 7.4 is released, leaving client-side disabled.
Then a year from now (June 2017) all protocol 1 code will be removed.
Beastie Bits
Last day to get your BSDNow Shirts! Order now, wear at BSDCan! (https://teespring.com/bsdnow)
Move local government (Austin TX) from Microsoft Windows (incl. Office) to Linux and/or PC-BSD (https://github.com/atxhack4change/2016-project-proposals/issues/15)
Plan9 boot camp is back... and already at capacity. Another opportunity may come in September (http://lists.nycbug.org/pipermail/talk/2016-May/016642.html)
Smaller is better - building an openbsd based router (https://functionallyparanoid.com/2016/04/22/smaller-is-better/)
Baby Unix (https://i.redditmedia.com/KAjSscL9XOUdpIEWBQF1qi3QMr7zWgeETzQM6m3B4mY.jpg?w=1024&s=e8c08a7d4c4cea0256adb69b1e7c1887)
Security Update for FreeBSD (https://security.freebsd.org/advisories/FreeBSD-SA-16:19.sendmsg.asc) & Another security update for FreeBSD (https://security.freebsd.org/advisories/FreeBSD-SA-16:18.atkbd.asc)
Feedback/Questions
Eric - The iX experience (http://pastebin.com/ZknTuKGv)
Mike - Building Ports (http://pastebin.com/M760ZmHQ)
David - ZFS Backups (http://pastebin.com/Pi0AFghV)
James - BSD VPS (http://pastebin.com/EQ7envez)
Rich - ZFS Followup (http://pastebin.com/p0HPDisH)
***
| |||
| 29 Dec 2022 | 487: EuroBSDcon Interviews Pt. 2 | 00:34:19 | |
This year end episode of BSDNow features a trip report to EuroBSDcon by Mr. BSD.tv, as well as an interview with FreeBSD committer John Baldwin. Happy New Year, 2023!
NOTES***
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
EuroBSDCon 2022 Trip Report (https://freebsdfoundation.org/blog/eurobsdcon-2022-trip-report-patrick-mcevoy/)
Interview 3 - John Baldwin - email@email (mailto:email@email) / @twitter (https://twitter.com/user)
Interview topic
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 24 Dec 2014 | 69: Under the Ports Tree | 01:12:33 | |
It's a special holiday episode! We asked you guys in the audience to send in the tale of how you first got into BSD, and we're going to share those with everyone today. We'll also be playing two bonus mini-interviews, so get comfy by the fire and listen to some BSD Now - the place to B.. SD.
This episode was brought to you by
Special segment
How our viewers got into BSD
Jason's story (text (http://slexy.org/view/s207hi9pTo))
bsdx's story (text (http://slexy.org/view/s20cmh0anD))
David's story (text (http://slexy.org/view/s21r4AL53g))
Brad's story (text (http://slexy.org/view/s2OqEie53V))
Reese's story (video)
Bryan's story (video)
Pete's story (text (http://slexy.org/view/s2ve2kfgW7))
Anders' story (text (http://slexy.org/view/s20eL5EYMv))
Guillermo's story (text (http://slexy.org/view/s20KRuIaks))
Jonathan's story (text (http://slexy.org/view/s20IFqrc7O))
Adam's story (text (http://slexy.org/view/s2FnnJH9zs))
Chris' story (text (http://slexy.org/view/s21GazXKH2))
Tigersharke's story (text (http://slexy.org/view/s2iJdLoxzZ))
Roller and Kandie's stories (text (http://slexy.org/view/s203RsddHG))
Uwe's story (text (http://slexy.org/view/s2gmB5VaS3))
Pascal's story (text (http://slexy.org/view/s2PWntJ7Tc)) and (image (https://i.imgur.com/ekXbDvb.jpg))
***
Interview - Erwin Lansing - erwin@freebsd.org (mailto:erwin@freebsd.org)
BSD in Europe, getting people involved
Interview - Cristina Vintila - @cristina_crow (https://twitter.com/cristina_crow)
BSD conferences
| |||
| 10 Aug 2016 | 154: Myths, Pi’s & Features, oh my! | 00:49:44 | |
This week on BSDNow, we are taking a look at a few different tutorials, including running your very own RPi web-server. (Come-on, you
This episode was brought to you by
Headlines
broken features aren't used (http://www.tedunangst.com/flak/post/broken-features-arent-used)
This post from TedU talks about the difficulty of removing features from an operating system
“One of the difficulties in removing a feature is identifying all the potential users. A feature here could be a program bundled with an operating system, or a command line option, or maybe just a function in a library. If we remove a feature, users that depend on it will be sad. Unfortunately, absence of evidence is not evidence of absence. I’ve never heard of anybody running ls -p but it’s not impossible that somebody does.”
“The reasons why we want to remove an existing feature can vary. Sometimes it’s old code that interferes with maintenance. Sometimes a nearly complete rewrite can improve performance. In other cases, the feature in question is really more of a misfeature. It may have security implications, where the existence of the feature can be used to facilitate the exploitation of other vulnerabilities, and removing the feature will help mitigate the exploit.”
“There’s no general test that can be used, but there is one test that works in many cases. Test that the feature works. If the feature doesn’t work, that’s compelling evidence that nobody is using it, because nobody can be using it. You don’t need to fix it. You can just remove it.”
He makes some interesting comments about exhaustive unit tests and the push to keep everything working all the time. If you never break anything to see if someone complains, how do you know if it is still being used?
***
A Raspberry Pi FreeBSD Web Server (http://devio.us/~spacemonkey/raspberry_pi_freebsd_web_server_howto.html)
Looking at a super-low power solution to host some webpages? If so, we have the tutorial for you.
Specifically a walkthrough of getting FreeBSD up on a Pi, and setting up nginx, OpenNTPD, LibreSSL and friends.
The walkthrough starts with grabbing a FreeBSD 11 snapshot for arm64 and doing the initial setup process to get to a bootable FreeBSD system.
If you are an extreme noob, not to fear. The tutorial walks you through setting up usernames, timezones, even a larger /tmp directory on your new MiniBSD setup.
The tedious part comes to play during the setup of packages. The author walks us through setting up LibreSSL and various other packages via ports (Since LibreSSL isn’t the default in FreeBSD). This will take some time to compile on your humble RPi device. (Go make a sandwich, walk the dog, fix the gutters, etc)
When it’s all said and done, you’ll end up with a secure little web-server that you’ve configured all by yourself! (Wondering what the word-press performance would be like on that box)
***
Uber switches from PostgreSQL back to MySQL (https://eng.uber.com/mysql-migration/)
We often hear success stories of people switching to PostgreSQL and getting huge performance gains, but this stories is the reverse
Uber’s engineering team has switched back to MySQL, because for their specific workload and design, MySQL’s innodb has better performance
Of course, it is not just vanilla MySQL, but “Schemaless”, a sharding system that sits on top of MySQL
The article goes into detail about the on-disk format used by Postgres, and the specific shortcomings that Uber encountered
Uber admits that all of its testing was against the older PostgreSQL 9.2, but one of their complaints is about having difficulty upgrading
“We started out with Postgres 9.1 and successfully completed the upgrade process to move to Postgres 9.2. However, the process took so many hours that we couldn’t afford to do the process again. By the time Postgres 9.3 came out, Uber’s growth increased our dataset substantially, so the upgrade would have been even lengthier. For this reason, our legacy Postgres instances run Postgres 9.2 to this day, even though the current Postgres GA release is 9.5.”
There is a followup, from the Postgres side
“Why we lost Uber as a user” (https://www.postgresql.org/message-id/5797D5A1.5030009%40agliodbs.com)
This thread goes into detail about the specific types of problematic queries that Uber was using
“The Uber guy is right that InnoDB handles this better as long as you don't touch the primary key (primary key updates in InnoDB are really bad)”
“This is a common problem case we don't have an answer for yet.”
The thread then goes on to discuss possibly supporting a “pluggable heap storage layer”, to allow different workloads to use different on-disk formats for best performance
***
Getting started with GhostBSD and FreeBSD
Part 1 (http://unsolicitedbutoffered.blogspot.com/2016/07/getting-started-with-ghostbsd-and.html)
Part 2 (http://unsolicitedbutoffered.blogspot.com/2016/07/getting-started-with-ghostbsd-and_31.html)
Part 3 (http://unsolicitedbutoffered.blogspot.com/2016/08/getting-started-with-ghostbsd-and.html)
Part 4 (http://unsolicitedbutoffered.blogspot.com/2016/08/getting-started-with-ghostbsd-and_2.html)
In what may be our first GhostBSD tutorial, we have a nice walkthrough on the initial getting started with it.
For those who don’t know, GhostBSD provides a nice XFCE or Mate desktop out of box, and still supports 32bit installs for those who want to keep that older hardware running.
The walkthough takes us through the process of grabbing GhostBSD images and getting the installer up and running via bootable USB stick.
Once booted, the graphical installer is straight-forward and short, allowing you to get the bits on disk as quickly as possible. (The actual installation took around 45 Minutes on an old Toshiba NB520)
The author then takes us on a tour of some of GhostBSD’s out-of-box bundled applications (Along with XFCE) and how it compares to similar Linux setups.
Lastly covered is the setup of Wireless (The manual way with WPA supplicant, since the GUI tool appeared to not work in this particular case)
All in all a good walkthrough, especially if you’ve not seen GhostBSD in action before, the screenshots are very informative!
***
News Roundup
Steam on FreeBSD 11-CURRENT (https://github.com/SteamOnFreeBSD/SteamOnFreeBSD)
Steam on FreeBSD. Yes, we’ve heard of setups using WINE, but what about running the Linux binaries natively?
Well you are in luck. We have a github project that details getting the Linux native client up and running on a FreeBSD 11-CURRENT system.
This github project is rather mysterious, with only the instructions to download a pre-packaged steam.txz file, extract and run the provided install.sh script.
Curious I inspected some of the scripts, the installer.sh is fairly straight-forward, but does some ‘non-standard’ freebsd things, like fetching packages and extracting specific files/libaries into a new /compat/ubuntu directory.
After that, it goes through a huge list of debian/ubuntu packages, also throwing them into the aforementioned ubuntu directory.
At runtime, the wrapper script ensures that various linux compat file-systems are mounted in the correct location, then proceeds to run steam with some LDLIBRARYFLAGS set from the users .local/share/steam directory.
A tad scary if honest, however it is a neat PoC to see Steam working on FreeBSD. Hopefully somebody can turn this into a more traditional package which can be easily removed / cleaned up afterwards.
***
How to run Enlightenment on OpenBSD (http://enform.haxlab.org/)
Are you an enlightenment fan? Be honest, we know there are quite a few of you out there!
If so, we have a tutorial for you today, which talks about how to run E on OpenBSD.
The process is pretty easy, but some steps might be overlooked if you are new to OpenBSD or don’t know how to tune / compile things on your own.
It starts out with adjusting some sysctl’s for better tuning that works on an E based desktop.
Next is installing from package some pre-reqs that will enable us to build E from source.
After that, we need to download and install EFL from github, and the autogen / gmake commands are helpfully provided for you.
Lastly the same is done for E itself, and TADA, E is installed and ready to go on your OpenBSD system. If you do this right, should only take 5-10 minutes to be up and running.
***
Myths about FreeBSD (https://wiki.freebsd.org/Myths)
Over on the FreeBSD wiki, we have a new “Myths” page which we’ve never highlighted on the show before.
First up, and one I’ve very familiar with, is the usual “FreeBSD is only for Servers and not Desktops”, along with a good rebuttal about what it does offer and mention of projects such as PC-BSD which do it also.
Another prevalent one is the “FreeBSD has a closed development model”, which is easily refuted:
FreeBSD has over 400 developers around the world who have commit access to the repository. Many of these are willing to commit patches from third parties. If you want to get an idea of the number of patches that have been committed on behalf of other developers, then search for 'Submitted by' in the commit logs. At the time of writing, this is just under twenty thousand, or about ten percent of all commits. After having a few patches accepted, regular contributors are usually encouraged to apply for commit access.
Another one that we are still hearing (Although it is less and less now) was the common “FreeBSD makes me compile everything from source”. Listeners of this show will know that pkg has pretty much made this irrelevant in recent years. However the option to compile yourself from source still exists, but most users won’t find this ever necessary.
A good list, with many more items on it than we’ve mentioned here. Take a look, you might find something there you’ve heard in the wild, or maybe even thought yourself at one point!
FreeBSD Area51 testing repo, KDE 5 (https://gist.github.com/swills/8a9a9f8a529256f134830c2a1476db0c)
This github GIST from Steve Wills provides the instructions to enable the FreeBSD xorg teams unofficial ‘Area51’ testing repo on your machine
This gives you access to the new KDE 5
Users should obviously be careful testing early-access software, but bug reports are very welcome, and important to getting KDE 5 working well under FreeBSD
***
Beastie Bits
NetBSD Machines at Open Source Conference 2016 Kyoto (http://mail-index.netbsd.org/netbsd-advocacy/2016/08/01/msg000712.html)
How to install htop on pfSense (http://www.cyberciti.biz/faq/pfsense-install-htop-using-pkg-command/)
The first ever patch for #Chromium on #NetBSD has been committed! (https://twitter.com/krytarowski/status/759909139300491265)
CharmBug - Adventures in Hardened BSD - August 24th 2016 (http://www.meetup.com/CharmBUG/events/232474857/)
Feedback/Questions
Emanual - GoTTY (http://pastebin.com/q24sNcNY)
Joe - Disable Device (http://pastebin.com/iTkmpVr9)
Allen - ZFS Send/Recv (http://pastebin.com/bPHEP4Ya)
Mica - Adobe Replacements (http://pastebin.com/L0ttuNx3)
Robroy - RAID (http://pastebin.com/Dr92CGCU)
***
| |||
| 26 Aug 2015 | 104: Beverly Hills 25519 | 01:20:44 | |
Coming up this week on the show, we'll be talking with Damien Miller of the OpenSSH team. Their 7.0 release has some major changes, including phasing out older crypto and changing one of the defaults that might surprise you.
This episode was brought to you by
Headlines
EdgeRouter Lite, meet OpenBSD (http://www.tedunangst.com/flak/post/OpenBSD-on-ERL)
The ERL, much like the Raspberry Pi and a bunch of other cheap boards, is getting more and more popular as more things get ported to run on it
We've covered installing NetBSD and FreeBSD on them before, but OpenBSD has gotten a lot better support for them as well now (including the onboard storage in 5.8)
Ted Unangst got a hold of one recently and kindly wrote up some notes about installing and using OpenBSD on it
He covers doing a network install, getting the (slightly strange) bootloader working with u-boot and some final notes about the hardware
More discussion can be found on Hacker News (https://news.ycombinator.com/item?id=10079210) and various (https://www.reddit.com/r/openbsd/comments/3hgf2c) other (https://www.marc.info/?t=143974140500001&r=1&w=2) places (https://lobste.rs/s/acz9bu/openbsd_on_edgerouter_lite)
One thing to note (https://www.marc.info/?l=openbsd-misc&m=143991822827285&w=2) about these devices: because of their MIPS64 processor, they'll have weaker ASLR than X86 CPUs (and no W^X at all)
***
Design and Implementation of the FreeBSD Operating System interview (http://www.infoq.com/articles/freebsd-design-implementation-review)
For those who don't know, the "Design and Implementation of the FreeBSD Operating System" is a semi-recently-revived technical reference book for FreeBSD development
InfoQ has a review of the book up for anyone who might be interested, but they also have an interview the authors
"The book takes an approach to FreeBSD from inside out, starting with kernel services, then moving to process and memory management, I/O and devices, filesystems, IPC and network protocols, and finally system startup and shutdown. The book provides dense, technical information in a clear way, with lots of pseudo-code, diagrams, and tables to illustrate the main points."
Aside from detailing a few of the chapters, the interview covers who the book's target audience is, some history of the project, long-term support, some of the newer features and some general OS development topics
***
Path list parameter in OpenBSD tame (https://www.marc.info/?l=openbsd-cvs&m=144027474117290&w=2)
We've mentioned OpenBSD's relatively new "tame (https://marc.info/?l=openbsd-tech&m=143725996614627&w=2)" subsystem a couple times before: it's an easy-to-implement "self-containment" framework, allowing programs to have a reduced feature set mode with even less privileges
One of the early concerns from users of other process containment tools was that tame was too broad in the way it separated disk access - you could either read/write files or not, nothing in between
Now there's the option to create a whitelist of specific files and directories that your binary is allowed to access, giving a much finer-grained set of controls to developers
The next step is to add tame restraints to the OpenBSD userland utilities, which should probably be done by 5.9
More discussion can be found on Reddit (https://www.reddit.com/r/openbsd/comments/3i2lk7) and Hacker News (https://news.ycombinator.com/item?id=10104886)
***
FreeBSD & PC-BSD 10.2-RELEASE (https://www.freebsd.org/releases/10.2R/announce.html)
The FreeBSD team has released the second minor version bump to the 10.x branch, including all the fixes from 10-STABLE since 10.1 came out
The Linux compatibility layer has been updated to support CentOS 6, rather than the much older Fedora Core base used previously, and the DRM graphics code has been updated to match Linux 3.8.13
New installations (and newly-upgraded systems) will use the quarterly binary package set, rather than the rolling release model that most people are used to
A VXLAN driver was added, allowing you to create virtual LANs by encapsulating the ethernet frame in a UDP packet
The bhyve codebase is much newer, enabling support for AMD CPUs with SVM and AMD-V extensions
ARM and ARM64 code saw some fixes and improvements, including SMP support on a few specific boards and support for a few new boards
The bootloader now supports entering your GELI passphrase before loading the kernel in full disk encryption setups
In addition to assorted userland fixes and driver improvements, various third party tools in the base system were updated: resolvconf, ISC NTPd, netcat, file, unbound, OpenSSL, sendmail
Check the full release notes (https://www.freebsd.org/releases/10.2R/relnotes.html) for the rest of the details and changes
PC-BSD also followed with their 10.2-RELEASE (http://blog.pcbsd.org/2015/08/pc-bsd-10-2-release-now-available), sporting a few more additional features
***
Interview - Damien Miller - djm@openbsd.org (mailto:djm@openbsd.org) / @damienmiller (https://twitter.com/damienmiller)
OpenSSH: phasing out broken crypto, default cipher changes
News Roundup
NetBSD at Open Source Conference Shimane (https://mail-index.netbsd.org/netbsd-advocacy/2015/08/22/msg000692.html)
We weren't the only ones away at conferences last week - the Japanese NetBSD guys are always raiding one event or another
This time they had NetBSD running on some Sony NWS devices (MIPS-based)
JavaStations (https://en.wikipedia.org/wiki/JavaStation) were also on display - something we haven't ever seen before (made between 1996-2000)
***
BAFUG videos (https://www.youtube.com/watch?v=-XF20nitI90)
The Bay Area FreeBSD users group has been uploading some videos of their recent meetings
Devin Teske hosts the first one, discussing adding GELI support to the bootloader, including some video demonstrations of how it works
Shortly after beginning, Adrian Chadd takes over the conversation and they discuss various problems (and solutions) related to the bootloader - for example, how can we type encryption passwords with non-US keyboard layouts
In a second video (https://www.youtube.com/watch?v=49sPYHh473U), Jordan Hubbard and Kip Macy introduce "NeXTBSD aka FreeBSD X"
In it, they discuss their ideas of merging more Mac OS X features into FreeBSD (launchd to replace the init system, some APIs, etc)
People should record presentations at their BSD users groups and send them to us
***
L2TP over IPSEC on OpenBSD (http://frankgroeneveld.nl/2015/08/16/configuring-l2tp-over-ipsec-on-openbsd-for-mac-os-x-clients)
If you've got an OpenBSD box and some Mac OS X clients that need secure communications, surprise: they can work together pretty well
Using only the base tools in both operating systems, you can build a nice IPSEC setup for tunneling all your traffic
This guide specifically covers L2TP, using npppd and pre-shared keys
Server setup, client setup, firewall configuration and routing-related settings are all covered in detail
***
Reliable bare metal with TrueOS (http://www.tubsta.com/2015/08/reliable-bare-metal-server-using-trueosfreebsd)
Imagine a server version of PC-BSD with some useful utilities preinstalled - that's basically TrueOS
This article walks you through setting up a FreeBSD -CURRENT server (using TrueOS) to create a pretty solid backup solution
Most importantly, he also covers how to keep everything redundant and deal with hard drives failing
The author chose to go with the -CURRENT branch because of the delay between regular releases, and newer features not making their way to users as fast as he'd like
Another factor is that there are no binary snapshots of FreeBSD -CURRENT that can be easily used for in-place upgrades, but with TrueOS (and some other BSDs) there are
***
Kernel W^X on i386 (https://www.marc.info/?l=openbsd-cvs&m=144047868127049&w=2)
We mentioned some big W^X kernel changes in OpenBSD a while back (https://www.marc.info/?l=openbsd-tech&m=142120787308107&w=2), but the work was mainly for x86_64 CPU architecture (which makes sense; that's what most people run now)
Mike Larkin is back again, and isn't leaving the people with older hardware out, committing similar kernel work into the i386 platform now as well
Check out our interview with Mike (http://www.bsdnow.tv/episodes/2015_05_13-exclusive_disjunction) for some more background info on memory protections like W^X
***
Feedback/Questions
Markus writes in (http://slexy.org/view/s2iGoeYMyb)
Sean writes in (http://slexy.org/view/s21bIFfmUS)
Theo writes in (http://slexy.org/view/s21Hjm8Tsa)
***
| |||
| 22 Jun 2023 | 512: BSDNow Live! 9 bits of BSDNow - Just speak into the goat | 00:40:35 | |
Recorded at BSDCan 2023
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
BSDNow - The early years
BSDNow - Production Process
News Roundup
FreeBSD Devsummit
BSDCan
Beastie Bits
How you can help the show!
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Corey asks - what is the status of netbsd 10?
How have things changed in the bsds over the history of the show?
Announcement
As a final thing Allan would like to make an announcement:
- Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 03 Apr 2025 | 605: Fediverse Weather Service | 00:58:43 | |
FediMeteo: How a Tiny €4 FreeBSD VPS Became a Global Weather Service for Thousands, Core Infrastructure: Why You Need to Control Your NTP, Automatic Display switch for OpenBSD laptop, Using a 2013 Mac Pro as a FreeBSD Desktop, Some terminal frustrations, Copying all files of a directory, including hidden ones, with cp, You Should Use /tmp/ More, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
FediMeteo: How a Tiny €4 FreeBSD VPS Became a Global Weather Service for Thousands (https://it-notes.dragas.net/2025/02/26/fedimeteo-how-a-tiny-freebsd-vps-became-a-global-weather-service-for-thousands/)
Core Infrastructure: Why You Need to Control Your NTP (https://klarasystems.com/articles/core-infrastructure-why-you-need-to-control-your-ntp/?utm_source=BSD%20Now&utm_medium=Podcast)
News Roundup
Automatic Display switch for OpenBSD laptop (https://www.tumfatig.net/2024/automatic-display-switch-for-openbsd-laptop/)
Using a 2013 Mac Pro as a FreeBSD Desktop (https://forums.FreeBSD.org/threads/using-a-2013-mac-pro-as-a-freebsd-desktop.96805/)
Some terminal frustrations (https://jvns.ca/blog/2025/02/05/some-terminal-frustrations/)
Copying all files of a directory, including hidden ones, with cp (https://bhoot.dev/2025/cp-dot-copies-everything/)
You Should Use /tmp/ More (https://atthis.link/blog/2025/58671.html)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Tyler - Toms request (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/605/feedback/Tyler%20-%20Toms%20request.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 12 Nov 2014 | 63: A Man's man(1) | 01:37:43 | |
This time on the show, we've got an interview with Kristaps Džonsons, the creator of mandoc. He tells us how the project got started and what its current status is across the various BSDs. We also have a mini-tutorial on using PF to throttle bandwidth. This week's news, answers to your emails and even some cheesy mailing list gold, coming up on BSD Now - the place to B.. SD.
This episode was brought to you by
Headlines
Updates to FreeBSD's random(4) (https://svnweb.freebsd.org/base?view=revision&revision=273872)
FreeBSD's random device, which presents itself as "/dev/random" to users (https://news.ycombinator.com/item?id=8550457), has gotten a fairly major overhaul in -CURRENT
The CSPRNG (cryptographically secure pseudo-random number generator) algorithm, Yarrow, now has a new alternative called Fortuna
Yarrow is still the default for now, but Fortuna can be used with a kernel option (and will likely be the new default in 11.0-RELEASE)
Pluggable modules can now be written to add more sources of entropy
These changes are expected to make it in 11.0-RELEASE, but there hasn't been any mention of MFCing them to 10 or 9
***
OpenBSD Tor relays and network diversity (https://lists.torproject.org/pipermail/tor-relays/2014-November/005661.html)
We've talked about getting more BSD-based Tor nodes (http://lists.nycbug.org/mailman/listinfo/tor-bsd) a few times in previous episodes
The "tor-relays" mailing list has had some recent discussion about increasing diversity in the Tor network, specifically by adding more OpenBSD nodes
With the security features and attention to detail, it makes for an excellent dedicated Tor box
More and more adversaries are attacking Tor nodes, so having something that can withstand that will help the greater network at large
A few users are even saying they'll convert their Linux nodes to OpenBSD to help out
Check the archive for the full conversation, and maybe run a node yourself (http://www.bsdnow.tv/tutorials/tor) on any of the BSDs
The Tor wiki page on OpenBSD is pretty out of date (https://lists.torproject.org/pipermail/tor-dev/2014-November/007715.html) (nine years old!?) and uses the old pf syntax, maybe one of our listeners can modernize it
***
SSP now default for FreeBSD ports (https://lists.freebsd.org/pipermail/freebsd-ports/2014-November/096344.html)
SSP, or Stack Smashing Protection (https://en.wikipedia.org/wiki/Buffer_overflow_protection), is an additional layer of protection against buffer overflows that the compiler can give to the binaries it produces
It's now enabled by default in FreeBSD's ports tree, and the pkgng packages will have it as well - but only for amd64 (all supported releases) and i386 (10.0-RELEASE or newer)
This will only apply to regular ports and binary packages, not the quarterly branch that only receives security updates
If you were using the temporary "new Xorg" or SSP package repositories instead of the default ones, you need to switch back over
NetBSD made this the default on i386 and amd64 two years ago (https://www.netbsd.org/releases/formal-6/NetBSD-6.0.html) and OpenBSD made this the default on all architectures twelve years ago (https://www.marc.info/?l=openbsd-cvs&m=103881967909595&w=2)
Next time you rebuild your ports, things should be automatically hardened without any extra steps or configuration needed
***
Building an OpenBSD firewall and router (https://www.reddit.com/r/BSD/comments/2ld0yw/building_an_openbsd_firewall_and_router/)
While we've discussed the software and configuration of an OpenBSD router, this Reddit thread focuses more on the hardware side
The OP lists some of his potential choices, but was originally looking for something a bit cheaper than a Soekris
Most agree that, if it's for a business especially, it's worth the extra money to go with something that's well known in the BSD community
They also list a few other popular alternatives: ALIX or the APU series from PC Engines, some Supermicro boards, etc.
Through the comments, we also find out that QuakeCon runs OpenBSD on their network
Hopefully most of our listeners are running some kind of BSD as their gateway - try it out (http://www.bsdnow.tv/tutorials/openbsd-router) if you haven't already
***
Interview - Kristaps Džonsons - kristaps@bsd.lv (mailto:kristaps@bsd.lv)
Mandoc, historical man pages, various topics
Tutorial
Throttling bandwidth with PF (http://www.bsdnow.tv/tutorials/openbsd-router#queues)
News Roundup
NetBSD at Kansai Open Forum 2014 (https://mail-index.netbsd.org/netbsd-advocacy/2014/11/08/msg000672.html)
Japanese NetBSD users invade yet another conference, demonstrating that they can and will install NetBSD on everything
From a Raspberry Pi to SHARP Netwalkers to various luna68k devices, they had it all
As always, you can find lots of pictures in the trip report
***
Getting to know your portmgr lurkers (http://blogs.freebsdish.org/portmgr/2014/11/04/getting-to-know-your-portmgr-lurker-ak/)
The lovable "getting to know your portmgr" series makes its triumphant return
This time around, they interview Alex, one of the portmgr lurkers that joined just this month
"How would you describe yourself?" "Too lazy."
Another post (http://blogs.freebsdish.org/portmgr/2014/11/08/getting-to-know-your-portmgr-lurker-ehaupt/) includes a short interview with Emanuel, another new lurker
We discussed the portmgr lurkers initiative with Steve Wills a while back (http://www.bsdnow.tv/episodes/2014_10_01-the_daemons_apprentice)
***
NetBSD's ARM port gets SMP (https://blog.netbsd.org/tnf/entry/working_arm_multiprocessor_support)
The ARM port of NetBSD now has SMP support, allowing more than one CPU to be used
This blog post on the website has a list of supported boards: Banana Pi, Cubieboard 2, Cubietruck, Merrii Hummingbird A31, CUBOX-I and NITROGEN6X
NetBSD's release team is working on getting these changes into the 7 branch before 7.0 is released
There are also a few nice pictures in the article
***
A high performance mid-range NAS (http://pivotallabs.com/high-performing-mid-range-nas-server-part-2-performance-tuning-iscsi/)
This blog post is about FreeNAS and optimizing iSCSI performance
It talks about using mid-range hardware with FreeNAS and different tunables you can change to affect performance
There are some nice graphs and lots of detail if you're interested in tweaking some of your own settings
They conclude "there is no optimal configuration; rather, FreeNAS can be configured to suit a particular workload"
***
Feedback/Questions
Heto writes in (http://slexy.org/view/s2xGCUj8mC)
Brad writes in (http://slexy.org/view/s2SJ8xppDJ)
Tyler writes in (http://slexy.org/view/s20Ktl6BMk)
Tim writes in (http://slexy.org/view/s2AsrxU0ZQ)
Brad writes in (http://slexy.org/view/s21yn0xLv2)
***
Mailing List Gold
Suspicious contributions (https://www.marc.info/?t=141379917200003&r=1&w=2)
La puissance du fromage (https://www.marc.info/?l=openbsd-cvs&m=141538800019451&w=2)
Nothing unusual here (https://mail-index.netbsd.org/tech-ports/2002/07/05/0000.html)
***
| |||
| 27 Jun 2024 | 565: Secure by default | 00:51:29 | |
NetBSD 10 on a Pinebook Pro, OpenBSD extreme privacy setup, Version 256 of systemd boasts '42% less Unix philosophy', Posix.1 2024 is out, Blocking Access From or to Specific Countries Using FreeBSD and Pf, and more.
Date: 2024.06.17
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
NetBSD 10 on a Pinebook Pro (https://www.idatum.net/netbsd-10-on-a-pinebook-pro-laptop.html)
OpenBSD extreme privacy setup (https://dataswamp.org/~solene/2024-06-08-openbsd-privacy-setup.html)
News Roundup
Version 256 of systemd boasts '42% less Unix philosophy' (https://www.theregister.com/2024/06/13/version_256_systemd/)
Posix.1 2024 is out (https://ieeexplore.ieee.org/document/10555529)
Blocking Access From or to Specific Countries Using FreeBSD and Pf (https://it-notes.dragas.net/2024/06/16/freebsd-blocking-country-access/)
Beastie Bits
BSD User Group Düsseldorf Juli 2024 (https://www.meetup.com/de-DE/bsd-user-group-dusseldorf-bsd-nrw/events/301557512/)
Another cool UNIX workstation, that was never released (https://www.reddit.com/r/unix/comments/1dd60re/another_cool_unix_workstation_that_was_never/)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 26 Feb 2014 | 26: Port Authority | 01:31:05 | |
On today's show we have an interview with Joe Marcus Clark, one of the original portmgr members in FreeBSD, and one of the key GNOME porters. Keeping along with that topic, we have a FreeBSD ports tutorial for you as well. The latest news and answers to your BSD questions, right here on BSD Now - the place to B.. SD.
This episode was brought to you by
Headlines
Tailoring OpenBSD for an old, strange computer (http://multixden.blogspot.com/2014/02/tailoring-openbsd-for-old-strange.html)
The author of this article had an OmniBook 800CT (http://hpmuseum.net/display_item.php?hw=233), which comes with a pop-out mouse, black and white display, 32MB of RAM and a 133MHz CPU
Obviously he had to install some kind of BSD on it!
This post goes through all his efforts of trimming down OpenBSD to work on such a limited device
He goes through the trial and error of "compile, break it, rebuild, try again"
After cutting a lot out from the kernel, saving a precious megabyte here and there, he eventually gets it working
***
pkgsrcCon and BSDCan (http://www.pkgsrc.org/pkgsrcCon/2014/)
pkgsrccon is "a technical conference for people working on the NetBSD Packages Collection, focusing on existing technologies, research projects, and works-in-progress in pkgsrc infrastructure"
This year it will be on June 21st and 22nd
The schedule (http://www.pkgsrc.org/pkgsrcCon/2014/schedule.html) is still being worked out, so if you want to give a talk, submit it
BSDCan's schedule (https://www.bsdcan.org/2014/schedule/events.en.html) was also announced
We'll be having presentations about ARM on NetBSD and FreeBSD, PF on OpenBSD, Capsicum and casperd, ASLR in FreeBSD, more about migrating from Linux to BSD, FreeNAS stuff and much more
Kris' presentation was accepted!
Tons of topics, look forward to the recorded versions of all of them hopefully!
***
Two factor auth with pushover (http://www.tedunangst.com/flak/post/login-pushover)
A new write-up from our friend Ted Unangst (http://www.bsdnow.tv/episodes/2014_02_05-time_signatures)
Pushover is "a web hook to smartphone push notification gateway" - you sent a POST to a web server and it sends a code to your phone
His post goes through the steps of editing your login.conf and setting it all up to work
Now you can get a two factor authenticated login for ssh!
***
The status of GNOME 3 on BSD (http://undeadly.org/cgi?action=article&sid=20140219085851)
It's no secret that the GNOME team is a Linux-obsessed bunch, almost to the point of being hostile towards other operating systems
OpenBSD keeps their GNOME 3 ports up to date very well, and Antoine Jacoutot writes about his work on that and how easy it is to use
This post goes through the process of how simple it is to get GNOME 3 set up on OpenBSD and even includes a screencast (https://www.bsdfrog.org/tmp/undeadly-gnome.webm)
A few recent (http://blogs.gnome.org/mclasen/2014/02/19/on-portability/) posts (http://blogs.gnome.org/desrt/2014/02/19/on-portability/) from some GNOME developers show that they're finally working with the BSD guys to improve portability
The FreeBSD and OpenBSD teams are working together to bring the latest GNOME to all of us - it's a beautiful thing
This goes right along with our interview today!
***
Interview - Joe Marcus Clark - marcus@freebsd.org (mailto:marcus@freebsd.org)
The life and daily activities of portmgr, GNOME 3, Tinderbox, portlint, various topics
Tutorial
The FreeBSD Ports Collection (http://www.bsdnow.tv/tutorials/ports)
News Roundup
DragonflyBSD 3.8 goals and 3.6.1 release (http://bugs.dragonflybsd.org/versions/4)
The Dragonfly team is thinking about what should be in version 3.8
On their bug tracker, it lists some of the things they'd like to get done before then
In the meantime, 3.6.1 (http://lists.dragonflybsd.org/pipermail/commits/2014-February/199294.html) was released with lots of bugfixes
***
NYCBSDCon 2014 wrap-up piece (http://www.informit.com/blogs/blog.aspx?uk=NYCBSDCon-2014-Rocked-a-Cold-February-Weekend)
We've got a nice wrap-up titled "NYCBSDCon 2014 Heats Up a Cold Winter Weekend"
The author also interviews GNN (http://www.bsdnow.tv/episodes/2014_01_29-journaled_news_updates) about the conference
There's even a little "beginner introduction" to BSD segment
Includes a mention of the recently-launched journal and lots of pictures from the event
***
FreeBSD and Linux, a comparative analysis (https://www.youtube.com/watch?&v=5mv_oKFzACM#t=418)
GNN in yet another story - he gave a presentation at the NYLUG about the differences between FreeBSD and Linux
He mentions the history of BSD, the patch set and 386BSD, the lawsuit, philosophy and license differences, a complete system vs "distros," development models, BSD-only features and technologies, how to become a committer, overall comparisons, different hats and roles, the different bsds and their goals and actual code differences
Serves as a good introduction you can show your Linux friends
***
PCBSD CFT and weekly digest (http://blog.pcbsd.org/2014/02/call-for-testers-new-major-upgrade-methodology/)
Upgrade tools have gotten a major rewrite
You have to help test it, there is no choice! Read more here (http://blog.pcbsd.org/2014/02/pc-bsd-weekly-feature-digest-18/)
How dare Kris be "unimpressed with" freebsd-update and pkgng!?
Various updates and fixes
***
Feedback/Questions
Jeffrey writes in (http://slexy.org/view/s213KxUdVj)
Shane writes in (http://slexy.org/view/s20lwkjLVK)
Ferdinand writes in (http://slexy.org/view/s21DqJs77g)
Curtis writes in (http://slexy.org/view/s20eXKEqJc)
Clint writes in (http://slexy.org/view/s21XMVFuVu)
Peter writes in (http://slexy.org/view/s20Xk05MHe)
***
| |||
| 05 Jan 2023 | 488: Old ping(8) bug | 00:35:25 | |
Finding a 24 year old bug in ping(8), The Role of Operating Systems in IOT, Authentication gateway with SSH on OpenBSD, FreeBSD 12.4 is out, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Fuzzing ping(8) … and finding a 24 year old bug (https://tlakh.xyz/fuzzing-ping.html)
The Role of Operating Systems in IOT (https://klarasystems.com/articles/the-role-of-operating-systems-in-iot/)
News Roundup
Authentication gateway with SSH on OpenBSD (https://dataswamp.org/~solene/2022-12-01-openbsd-authpf.html)
FreeBSD 12.4 is out (https://lists.freebsd.org/archives/freebsd-announce/2022-December/000059.html)
Beastie Bits
Vagrant FreeBSD Boxbuilder (https://github.com/punktDe/vagrant-freebsd-boxbuilder)
LibreSSL 3.7.0 Released (https://undeadly.org/cgi?action=article;sid=20221212183516)
OPNsense 22.7.9 released (https://opnsense.org/opnsense-22-7-9-released)
BIOS Memory Map for vmd(8) Rewrite in Progress (https://undeadly.org/cgi?action=article;sid=20221211164822)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
***
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 16 Jun 2022 | 459: NetBSD Kernel benchmark | 00:54:05 | |
Evaluating FreeBSD CURRENT for Production Use, Time Machine-like Backups on OpenBSD, FreeBSD on the Graviton 3, Compiling the NetBSD kernel as a benchmark, Network Management with the OpenBSD Packet Filter Toolset from BSDCan 2022, Hardware Detection & Diagnostics for New FreeBSD Users, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Evaluating FreeBSD CURRENT for Production Use (https://klarasystems.com/articles/evaluating-freebsd-current-for-production-use/)
Time Machine like Backups on OpenBSD (https://xosc.org/timemachine.html)
News Roundup
FreeBSD on the Graviton 3 (https://www.daemonology.net/blog/2022-05-23-FreeBSD-Graviton-3.html)
Compiling the NetBSD kernel as a benchmark (https://blog.anotherhomepage.org/post/2022/05/25/Compiling-the-NetBSD-kernel-as-a-benchmark/)
Network Management with the OpenBSD Packet Filter Toolset from BSDCan 2022 (http://undeadly.org/cgi?action=article;sid=20220607112236)
Hardware Detection & Diagnostics for New FreeBSD Users & PCs (https://forums.FreeBSD.org/threads/hardware-detection-diagnostics-for-new-freebsd-users-pcs.84596/)
Beastie Bits
• [NetBSD - Announcing Google Summer of Code 2022 projects](https://blog.netbsd.org/tnf/entry/announcing_google_summer_of_code3)
• [Welcome FreeBSD Google Summer of Code Participants](https://freebsdfoundation.org/blog/welcome-freebsd-google-summer-of-code-participants/)
• [Network from Scratch](https://www.networksfromscratch.com)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 28 Apr 2022 | 452: The unknown hackers | 00:46:29 | |
The unknown hackers, Papers we love to read, Dual Boot Homelab in The Bedroom by the bed testbed, OpenSSH 9.0 released, OS battle: OpenBSD vs. NixOS, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
The unknown hackers (https://www.salon.com/2000/05/17/386bsd/)
Bill Jolitz passed away in March 2022 (https://minnie.tuhs.org/pipermail/tuhs/2022-April/025643.html)
***
FreeBSD Documentation: Papers We Love To Read (https://klarasystems.com/articles/freebsd-documentation-papers-we-love-to-read/)
News Roundup
FreeBSD/Ubuntu Dual Boot Homelab in The Bedroom by the bed testbed (https://adventurist.me/posts/00307)
OpenSSH 9.0 has been released (https://www.openssh.com/txt/release-9.0)
Operating systems battle: OpenBSD vs NixOS (https://dataswamp.org/~solene/2022-04-18-openbsd-vs-nixos.html)
Beastie Bits
Celebrating 50 years of the Unix Operating System (https://www.reddit.com/r/BSD/comments/u4t25c/celebrating_50_years_of_the_unix_operating_system/)
Kickstarter Campaign Results (https://mwl.io/archives/13627)
FreeBSD Virtualization Series (https://productionwithscissors.run/freebsd-virtualization-series/)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Jeff - ZFS checksum repair (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/452/feedback/Jeff%20-%20ZFS%20checksum%20repair.md)
Nelson - General Thanks (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/452/feedback/Nelson%20-%20General%20Thanks.md)
Sam - FOSS Power Support (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/452/feedback/Sam%20-%20FOSS%20Power%20Support.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 17 Oct 2019 | 320: Codebase: Neck Deep | 00:56:41 | |
Headlines
FreeBSD and custom firmware on the Google Pixelbook (https://unrelenting.technology/articles/FreeBSD-and-custom-firmware-on-the-Google-Pixelbook)
FreeBSD and custom firmware on the Google Pixelbook
Back in 2015, I jumped on the ThinkPad bandwagon by getting an X240 to run FreeBSD on. Unlike most people in the ThinkPad crowd, I actually liked the clickpad and didn\u2019t use the trackpoint much. But this summer I\u2019ve decided that it was time for something newer. I wanted something..
lighter and thinner (ha, turns out this is actually important, I got tired of carrying a T H I C C laptop - Apple was right all along);
with a 3:2 display (why is Lenovo making these Serious Work\u2122 laptops 16:9 in the first place?? 16:9 is awful in below-13-inch sizes especially);
with a HiDPI display (and ideally with a good size for exact 2x scaling instead of fractional);
with USB-C ports;
without a dGPU, especially without an NVIDIA GPU;
assembled with screws and not glue (I don\u2019t necessarily need expansion and stuff in a laptop all that much, but being able to replace the battery without dealing with a glued chassis is good);
supported by FreeBSD of course (\u201csome development required\u201d is okay but I\u2019m not going to write big drivers);
how about something with open source firmware, that would be fun.
I was considering a ThinkPad X1 Carbon from an old generation - the one from the same year as the X230 is corebootable, so that\u2019s fun. But going back in processor generations just doesn\u2019t feel great. I want something more efficient, not less!
And then I discovered the Pixelbook. Other than the big huge large bezels around the screen, I liked everything about it. Thin aluminum design, a 3:2 HiDPI screen, rubber palm rests (why isn\u2019t every laptop ever doing that?!), the \u201cconvertibleness\u201d (flip the screen around to turn it into.. something rather big for a tablet, but it is useful actually), a Wacom touchscreen that supports a pen, mostly reasonable hardware (Intel Wi-Fi), and that famous coreboot support (Chromebooks\u2019 stock firmware is coreboot + depthcharge).
So here it is, my new laptop, a Google Pixelbook.
Conclusion
Pixelbook, FreeBSD, coreboot, EDK2 good.
Seriously, I have no big words to say, other than just recommending this laptop to FOSS enthusiasts :)
Porting NetBSD to the AMD x86-64: a case study in OS portability (https://www.usenix.org/legacy/publications/library/proceedings/bsdcon02/full_papers/linden/linden_html/index.html)
Abstract
NetBSD is known as a very portable operating system, currently running on 44 different architectures (12 different types of CPU). This paper takes a look at what has been done to make it portable, and how this has decreased the amount of effort needed to port NetBSD to a new architecture. The new AMD x86-64 architecture, of which the specifications were published at the end of 2000, with hardware to follow in 2002, is used as an example.
Portability
Supporting multiple platforms was a primary goal of the NetBSD project from the start. As NetBSD was ported to more and more platforms, the NetBSD kernel code was adapted to become more portable along the way.
General
Generally, code is shared between ports as much as possible. In NetBSD, it should always be considered if the code can be assumed to be useful on other architectures, present or future. If so, it is machine-independent and put it in an appropriate place in the source tree. When writing code that is intended to be machine-independent, and it contains conditional preprocessor statements depending on the architecture, then the code is likely wrong, or an extra abstraction layer is needed to get rid of these statements.
Types
Assumptions about the size of any type are not made. Assumptions made about type sizes on 32-bit platforms were a large problem when 64-bit platforms came around. Most of the problems of this kind had to be dealt with when NetBSD was ported to the DEC Alpha in 1994. A variation on this problem had to be dealt with with the UltraSPARC (sparc64) port in 1998, which is 64-bit, but big endian (vs. the little-endianness of the Alpha). When interacting with datastructures of a fixed size, such as on-disk metadata for filesystems, or datastructures directly interpreted by device hardware, explicitly sized types are used, such as uint32t, int8t, etc.
Conclusions and future work
The port of NetBSD to AMD's x86-64 architecture was done in six weeks, which confirms NetBSD's reputation as being a very portable operating system. One week was spent setting up the cross-toolchain and reading the x86-64 specifications, three weeks were spent writing the kernel code, one week was spent writing the userspace code, and one week testing and debugging it all. No problems were observed in any of the machine-independent parts of the kernel during test runs; all (simulated) device drivers, file systems, etc, worked without modification.
News Roundup
ZFS performance really does degrade as you approach quota limits (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSFullQuotaPerformanceIssue)
Every so often (currently monthly), there is an "OpenZFS leadership meeting". What this really means is 'lead developers from the various ZFS implementations get together to talk about things'. Announcements and meeting notes from these meetings get sent out to various mailing lists, including the ZFS on Linux ones.
In the September meeting notes, I read a very interesting (to me) agenda item:
Relax quota semantics for improved performance (Allan Jude)
Problem: As you approach quotas, ZFS performance degrades.
Proposal: Can we have a property like quota-policy=strict or loose, where we can optionally allow ZFS to run over the quota as long as performance is not decreased.
This is very interesting to me because of two reasons. First, in the past we have definitely seen significant problems on our OmniOS machines, both when an entire pool hits a quota limit and when a single filesystem hits a refquota limit. It's nice to know that this wasn't just our imagination and that there is a real issue here. Even better, it might someday be improved (and perhaps in a way that we can use at least some of the time).
Second, any number of people here run very close to and sometimes at the quota limits of both filesystems and pools, fundamentally because people aren't willing to buy more space. We have in the past assumed that this was relatively harmless and would only make people run out of space. If this is a known issue that causes serious performance degradation, well, I don't know if there's anything we can do, but at least we're going to have to think about it and maybe push harder at people. The first step will have to be learning the details of what's going on at the ZFS level to cause the slowdown. (It's apparently similar to what happens when the pool is almost full, but I don't know the specifics of that either.)
With that said, we don't seem to have seen clear adverse effects on our Linux fileservers, and they've definitely run into quota limits (repeatedly). One possible reason for this is that having lots of RAM and SSDs makes the effects mostly go away. Another possible reason is that we haven't been looking closely enough to see that we're experiencing global slowdowns that correlate to filesystems hitting quota limits. We've had issues before with somewhat subtle slowdowns that we didn't understand (cf), so I can't discount that we're having it happen again.
Fixing up KA9Q-unix, or "neck deep in 30 year old codebases.." (http://adrianchadd.blogspot.com/2019/09/fixing-up-ka9q-unix-or-neck-deep-in-30.html)
I'll preface this by saying - yes, I'm still neck deep in FreeBSD's wifi stack and 802.11ac support, but it turns out it's slow work to fix 15 year old locking related issues that worked fine on 11abg cards, kinda worked ok on 11n cards, and are terrible for these 11ac cards. I'll .. get there.
Anyhoo, I've finally been mucking around with AX.25 packet radio. I've been wanting to do this since I was a teenager and found out about its existence, but back in high school and .. well, until a few years ago really .. I didn't have my amateur radio licence. But, now I do, and I've done a bunch of other stuff with a bunch of other radios. The main stumbling block? All my devices are either Apple products or run FreeBSD - and none of them have useful AX.25 stacks. The main stacks of choice these days run on Linux, Windows or are a full hardware TNC.
So yes, I was avoiding hacking on AX.25 stuff because there wasn't a BSD compatible AX.25 stack. I'm 40 now, leave me be.
But! A few weeks ago I found that someone was still running a packet BBS out of San Francisco. And amazingly, his local node ran on FreeBSD! It turns out Jeremy (KK6JJJ) ported both an old copy of KA9Q and N0ARY-BBS to run on FreeBSD! Cool!
I grabbed my 2m radio (which is already cabled up for digital modes), compiled up his KA9Q port, figured out how to get it to speak to Direwolf, and .. ok. Well, it worked. Kinda.
HAMMER2 and fsck for review (https://www.dragonflydigest.com/2019/09/24/23540.html)
HAMMER2 is Copy on Write, meaning changes are made to copies of existing data. This means operations are generally atomic and can survive a power outage, etc. (You should read up on it!) However, there\u2019s now a fsck command, useful if you want a report of data validity rather than any manual repair process.
[The return of startx(1) for non-root users with some caveats (https://undeadly.org/cgi?action=article;sid=20190917091236)
Mark Kettenis (kettenis@) has recently committed changes which restore a certain amount of startx(1)/xinit(1) functionality for non-root users. The commit messages explain the situation:
```
CVSROOT: /cvs
Module name: src
Changes by: kettenis@cvs.openbsd.org 2019/09/15 06:25:41
Modified files:
etc/etc.amd64 : fbtab
etc/etc.arm64 : fbtab
etc/etc.hppa : fbtab
etc/etc.i386 : fbtab
etc/etc.loongson: fbtab
etc/etc.luna88k: fbtab
etc/etc.macppc : fbtab
etc/etc.octeon : fbtab
etc/etc.sgi : fbtab
etc/etc.sparc64: fbtab
Log message:
Add ttyC4 to lost of devices to change when logging in on ttyC0 (and in some cases also the serial console) such that X can use it as its VT when running without root privileges.
ok jsg@, matthieu@
CVSROOT: /cvs
Module name: xenocara
Changes by: kettenis@cvs.openbsd.org 2019/09/15 06:31:08
Modified files:
xserver/hw/xfree86/common: xf86AutoConfig.c
Log message:
Add modesetting driver as a fall-back when appropriate such that we can use it when running without root privileges which prevents us from scanning the PCI bus.
This makes startx(1)/xinit(1) work again on modern systems with inteldrm(4), radeondrm(4) and amdgpu(4). In some cases this will result in using a different driver than with xenodm(4) which may expose issues (e.g. when we prefer the intel Xorg driver) or loss of acceleration (e.g. older cards supported by radeondrm(4)).
ok jsg@, matthieu@
```
Beastie Bits
ASCII table and history. Or, why does Ctrl+i insert a Tab in my terminal? (https://bestasciitable.com/)
Sourcehut makes BSD software better (https://sourcehut.org/blog/2019-09-12-sourcehut-makes-bsd-software-better/)
Chaosnet for Unx (https://github.com/LM-3/chaos)
The Vim-Inspired Editor with a Linguistic Twist (https://cosine.blue/2019-09-06-kakoune.html)
bhyvearm64: CPU and Memory Virtualization on Armv8.0-A (https://papers.freebsd.org/2019/bsdcan/elisei-bhyvearm64_cpu_and_memory_virtualization_on_armv8.0_a/)
DefCon25 - Are all BSD created Equally - A Survey of BSD Kernel vulnerabilities (https://www.youtube.com/watch?v=a2m56Yq-EIs)
Feedback/Questions
Tim - GSoC project ideas for pf rule syntax translation (http://dpaste.com/1RCSFK7#wrap)
Brad - Steam on FreeBSD (http://dpaste.com/2SKA9YB#wrap)
Ruslan - FreeBSD Quarterly Status Report - Q2 2019 (http://dpaste.com/0DQM3Q1)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 08 Aug 2019 | 310: My New Free NAS | 00:48:09 | |
OPNsense 19.7.1 is out, ZFS on Linux still has annoying issues with ARC size, Hammer2 is now default, NetBSD audio – an application perspective, new FreeNAS Mini, and more.
Headlines
OPNsense 19.7.1 (https://opnsense.org/opnsense-19-7-1-released/)
We do not wish to keep you from enjoying your summer time, but this
is a recommended security update enriched with reliability fixes for the
new 19.7 series. Of special note are performance improvements as well
as a fix for a longstanding NAT before IPsec limitation.
Full patch notes:
system: do not create automatic copies of existing gateways
system: do not translate empty tunables descriptions
system: remove unwanted form action tags
system: do not include Syslog-ng in rc.freebsd handler
system: fix manual system log stop/start/restart
system: scoped IPv6 "%" could confuse mwexecf(), use plain mwexec() instead
system: allow curl-based downloads to use both trusted and local authorities
system: fix group privilege print and correctly redirect after edit
system: use cached address list in referrer check
system: fix Syslog-ng search stats
firewall: HTML-escape dynamic entries to display aliases
firewall: display correct IP version in automatic rules
firewall: fix a warning while reading empty outbound rules configuration
firewall: skip illegal log lines in live log
interfaces: performance improvements for configurations with hundreds of interfaces
reporting: performance improvements for Python 3 NetFlow aggregator rewrite
dhcp: move advanced router advertisement options to correct config section
ipsec: replace global array access with function to ensure side-effect free boot
ipsec: change DPD action on start to "dpdaction = restart"
ipsec: remove already default "dpdaction = none" if not set
ipsec: use interface IP address in local ID when doing NAT before IPsec
web proxy: fix database reset for Squid 4 by replacing use of sslcrtd with securityfile_certgen
plugins: os-acme-client 1.24[1]
plugins: os-bind 1.6[2]
plugins: os-dnscrypt-proxy 1.5[3]
plugins: os-frr now restricts characters BGP prefix-list and route-maps[4]
plugins: os-google-cloud-sdk 1.0[5]
ports: curl 7.65.3[6]
ports: monit 5.26.0[7]
ports: openssh 8.0p1[8]
ports: php 7.2.20[9]
ports: python 3.7.4[10]
ports: sqlite 3.29.0[11]
ports: squid 4.8[12]
Stay safe and hydrated, Your OPNsense team
ZFS on Linux still has annoying issues with ARC size (https://utcc.utoronto.ca/~cks/space/blog/linux/ZFSOnLinuxARCShrinkage)
One of the frustrating things about operating ZFS on Linux is that the ARC size is critical but ZFS's auto-tuning of it is opaque and apparently prone to malfunctions, where your ARC will mysteriously shrink drastically and then stick there.
Linux's regular filesystem disk cache is very predictable; if you do disk IO, the cache will relentlessly grow to use all of your free memory. This sometimes disconcerts people when free reports that there's very little memory actually free, but at least you're getting value from your RAM. This is so reliable and regular that we generally don't think about 'is my system going to use all of my RAM as a disk cache', because the answer is always 'yes'. (The general filesystem cache is also called the page cache.)
This is unfortunately not the case with the ZFS ARC in ZFS on Linux (and it wasn't necessarily the case even on Solaris). ZFS has both a current size and a 'target size' for the ARC (called 'c' in ZFS statistics). When your system boots this target size starts out as the maximum allowed size for the ARC, but various events afterward can cause it to be reduced (which obviously limits the size of your ARC, since that's its purpose). In practice, this reduction in the target size is both pretty sticky and rather mysterious (as ZFS on Linux doesn't currently expose enough statistics to tell why your ARC target size shrunk in any particular case).
The net effect is that the ZFS ARC is not infrequently quite shy and hesitant about using memory, in stark contrast to Linux's normal filesystem cache. The default maximum ARC size starts out as only half of your RAM (unlike the regular filesystem cache, which will use all of it), and then it shrinks from there, sometimes very significantly, and once shrunk it only recovers slowly (if at all).
News Roundup
Hammer2 is now default (http://lists.dragonflybsd.org/pipermail/commits/2019-June/718989.html)
```
commit a49112761c919d42d405ec10252eb0553662c824
Author: Matthew Dillon | |||
| 30 Jan 2020 | 335: FreeBSD Down Under | 00:53:54 | |
Hyperbola Developer interview, why you should migrate from Linux to BSD, FreeBSD is an amazing OS, improving the ptrace(2) API in LLVM 10, First FreeBSD conference in Australia, and a guide to containers on FreeNAS.
Headlines
FreeBSD is an amazing operating System (https://www.unixsheikh.com/articles/freebsd-is-an-amazing-operating-system.html)
Update 2020-01-21: Since I wrote this article it got posted on Hacker News, Reddit and Lobster, and a few people have emailed me with comments. I have updated the article with comments where I have found it needed. As an important side note I would like to point out that I am not a FreeBSD developer, there may be things going on in the FreeBSD world that I know absolutely nothing about. I am also not glued to the FreeBSD developer mailing lists. I am not a FreeBSD "fanboy". I have been using GNU/Linux a ton more for the past two decades than FreeBSD, mainly due to hardware incompatibility (lacking or buggy drivers), and I love both Debian GNU/Linux and Arch Linux just as much as FreeBSD. However, I am concerned about the development of GNU/Linux as of late. Also this article is not about me trying to make anyone switch from something else to FreeBSD. It's about why I like FreeBSD and that I recommend you try it out if you're into messing with operating systems.
I think the year was late 1999 or mid 2000 when I one day was browsing computer books at my favorite bookshop and I discovered the book The Complete FreeBSD third edition from 1999 by Greg Lehey. With the book came 4 CD Roms with FreeBSD 3.3.
I had already familiarized myself with GNU/Linux in 1998, and I was in the process of migrating every server and desktop operating system away from Microsoft Windows, both at home and at my company, to GNU/Linux, initially Red Hat Linux and then later Debian GNU/Linux, which eventually became my favorite GNU/Linux distribution for many years.
When I first saw The Complete FreeBSD book by Greg Lehey I remember noticing the text on the front page that said, "The Free Version of Berkeley UNIX" and "Rock Solid Stability", and I was immediately intrigued! What was that all about? A free UNIX operating system! And rock solid stability? That sounded amazing.
Hyperbola Dev Interview (https://itsfoss.com/hyperbola-linux-bsd/)
In late December 2019, Hyperbola announced that they would be making major changes to their project. They have decided to drop the Linux kernel in favor of forking the OpenBSD kernel. This announcement only came months after Project Trident announced that they were going in the opposite direction (from BSD to Linux).
Hyperbola also plans to replace all software that is not GPL v3 compliant with new versions that are.
To get more insight into the future of their new project, I interviewed Andre, co-founder of Hyperbola.
News Roundup
Improving the ptrace(2) API and preparing for LLVM-10.0 (https://blog.netbsd.org/tnf/entry/improving_the_ptrace_2_api)
This month I have improved the NetBSD ptrace(2) API, removing one legacy interface with a few flaws and replacing it with two new calls with new features, and removing technical debt.
As LLVM 10.0 is branching now soon (Jan 15th 2020), I worked on proper support of the LLVM features for NetBSD 9.0 (today RC1) and NetBSD HEAD (future 10.0).
The first FreeBSD conference in Australia (https://rubenerd.com/the-first-freebsd-conference-in-australia/)
FreeBSD has existed as an operating system, project, and foundation for more than twenty years, and its earlier incantations have exited for far longer. The old guard have been developing code, porting software, and writing documentation for longer than I’ve existed. I’ve been using it for more than a decade for personal projects, and professionally for half that time.
While there are many prominent Australian FreeBSD contributors, sysadmins, and users, we’ve always had to venture overseas for conferences. We’re always told Australians are among the most ardent travellers, but I always wondered if we could do a domestic event as well.
And on Tuesday, we did! Deb Goodkin and the FreeBSD Foundation graciously organised and chaired a dedicated FreeBSD miniconf at the long-running linux.conf.au event held each year in a different city in Australia and New Zealand.
A practical guide to containers on FreeNAS for a depraved psychopath (https://medium.com/@andoriyu/a-practical-guide-to-containers-on-freenas-for-a-depraved-psychopath-c212203c0394)
This is a simple write-up to setup Docker on FreeNAS 11 or FreeBSD 11.
But muh jails?
You know that jails are dope and you know that jails are dope, yet no one else knows it. So here we are stuck with docker. Two years ago I would be the last person to recommend using docker, but a whole lot of things has changes past years…
So jails are dead then?
No, jails are still dope, but jails lack tools to manage them. Yes, there are a few tools, but they meant for hard-core FreeBSD users who used to suffering. Docker allows you to run applications without deep knowledge of application you’re running. It will also allow you to run applications that are not ported to FreeBSD.
Why you should migrate everything from Linux to BSD (https://www.unixsheikh.com/articles/why-you-should-migrate-everything-from-linux-to-bsd.html)
As an operating system GNU/Linux has become a real mess because of the fragmented nature of the project, the bloatware in the kernel, and because of the jerking around by commercial interests.
Response Should you migrate from Linux to BSD? It depends. (https://fediverse.blog/~/AllGoodThings/should-you-migrate-from-linux-to-bsd-it-depends)
Beastie Bits
Using the OpenBSD ports tree with dedicated users (https://dataswamp.org/~solene/2020-01-11-privsep.html)
broot on FreeBSD (https://vermaden.wordpress.com/2020/01/10/run-broot-on-freebsd/)
A Trip down Memory Lane (https://svnweb.freebsd.org/base/head/share/misc/bsd-family-tree?view=co)
Running syslog-ng in BastilleBSD (https://www.syslog-ng.com/community/b/blog/posts/running-syslog-ng-in-bastillebsd)
NASA : Using Software Packages in pkgsrc (https://www.nas.nasa.gov/hecc/support/kb/using-software-packages-in-pkgsrc_493.html)
Feedback/Questions
All of our questions this week were pretty technical in nature so I'm going to save those for the next episode so Allan can weigh in on them, since if we cover them now we're basically going to be deferring to Allan anyway.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 29 Dec 2016 | 174: 2016 Highlights | 02:55:34 | |
A look back at 2016
This episode was brought to you by
Links | |||
| 27 Dec 2017 | 226: SSL: Santa’s Syscall List | 02:07:41 | |
We read the FreeBSD Q3 status report, explore good and bad syscalls, list GOG Games for OpenBSD, and show you what devmatch can do.
This episode was brought to you by
Headlines
FreeBSD Q3 Status Report 2017 (https://lists.freebsd.org/pipermail/freebsd-announce/2017-December/001818.html)
FreeBSD Team Reports
FreeBSD Release Engineering Team
Ports Collection
The FreeBSD Core Team
The FreeBSD Foundation
Projects
FreeBSD CI
Kernel
Intel 10G iflib Driver Update
Intel iWARP Support
pNFS Server Plan B
Architectures
AMD Zen (family 17h) support
Userland Programs
Updates to GDB
Ports
FreeBSDDesktop
OpenJFX 8
Puppet
Documentation
Absolute FreeBSD, 3rd Edition
Manual Pages
Third-Party Projects
The nosh Project
####FreeBSD Foundation Q4 Update (https://www.freebsdfoundation.org/wp-content/uploads/2017/12/FreeBSD-Foundation-Q4-Update.pdf)
***
###11 syscalls that rock the world (https://www.cloudatomiclab.com/prosyscall/)
0. read
> You cannot go wrong with a read. You can barely EFAULT it! On Linux amd64 it is syscall zero. If all its arguments are zero it returns zero. Cool!
1. pipe
> The society for the preservation of historic calling conventions is very fond of pipe, as in many operating systems and architectures it preserves the fun feature of returning both of the file descriptors as return values. At least Linux MIPS does, and NetBSD does even on x86 and amd64. Multiple return values are making a comeback in languages like Lua and Go, but C has always had a bit of a funny thing about them, but they have long been supported in many calling conventions, so let us use them in syscalls! Well, one syscall.
2. kqueue
> When the world went all C10K on our ass, and scaleable polling was a thing, Linux went epoll, the BSDs went kqueue and Solaris went /dev/poll. The nicest interface was kqueue, while epoll is some mix of edge and level triggered semantics and design errors so bugs are still being found.
3. unshare
> Sounds like a selfish syscall, but this generous syscall call is the basis of Linux namespaces, allowing a process to isolate its resources. Containers are built from unshares.
4. setns
> If you liked unshare, its younger but cooler friend takes file descriptors for namespaces. Pass it down a unix socket to another process, or stash it for later, and do that namespace switching. All the best system calls take file descriptors.
5. execveat
> Despite its somewhat confusing name (FreeBSD has the saner fexecve, but other BSDs do not have support last time I checked), this syscall finally lets you execute a program just given a file descriptor for the file. I say finally, as Linux only implemented this in 3.19, which means it is hard to rely on it (yeah, stop using those stupid old kernels folks). Before that Glibc had a terrible userspace implementation that is basically useless. Perfect for creating sandboxes, as you can sandbox a program into a filesystem with nothing at all in, or with a totally controlled tree, by opening the file to execute before chroot or changing the namespace.
6. pdfork
> Too cool for Linux, you have to head out to FreeBSD for this one. Like fork, but you get a file descriptor for the process not a pid. Then you can throw it in the kqueue or send it to another process. Once you have tried process descriptors you will never go back.
7. signalfd
> You might detect a theme here, but if you have ever written traditional 1980s style signal handlers you know how much they suck. How about turning your signals into messages that you can read on, you guessed it, file descriptors. Like, usable.
8. wstat
> This one is from Plan 9. It does the opposite of stat and writes the same structure. Simples. Avoids having chmod, chown, rename, utime and so on, by the simple expedient of making the syscall symmetric. Why not?
9. clonefile
> The only cool syscall on OSX, and only supported on the new APFS filesystem. Copies whole files or directories on a single syscall using copy on write for all the data. Look on my works, copyfilerange and despair.
10. pledge
> The little sandbox that worked. OpenBSD only here, they managed to make a simple sandbox that was practical for real programs, like the base OpenBSD system. Capsicum form FreeBSD (and promised for Linux for years but no sign) is a lovely design, and gave us pdfork, but its still kind of difficult and intrusive to implement. Linux has, well, seccomp, LSMs, and still nothing that usable for the average program.
###Eleven syscalls that suck (https://www.cloudatomiclab.com/antisyscall/)
0. ioctl
> It can‘t decide if it‘s arguments are integers, strings, or some struct that is lost in the midst of time. Make up your mind! Plan 9 was invented to get rid of this.
1. fcntl
> Just like ioctl but for some different miscellaneous operations, because one miscelleny is not enough.
2. tuxcall
> Linux put a web server in the kernel! To win a benchmark contest with Microsoft! It had it‘s own syscall! My enum tux_reactions are YUK! Don‘t worry though, it was a distro patch (thanks Red Hat!) and never made it upstream, so only the man page and reserved number survive to taunt you and remind you that the path of the righteous is beset by premature optmization!
3. iosetup
> The Linux asynchronous IO syscalls are almost entirely useless! Almost nothing works! You have to use ODIRECT for a start. And then they still barely work! They have one use, benchmarking SSDs, to show what speed you could get if only there was a usable API. Want async IO in kernel? Use Windows!
4. stat, and its friends and relatives
> Yes this one is useful, but can you find the data structure it uses? We have oldstat, oldfstat, ustat, oldlstat, statfs, fstatfs, stat, lstat, fstat, stat64, lstat64, fstat64, statfs64, fstatfs64, fstatat64 for stating files and links and filesystems in Linux. A new bunch will be along soon for Y2038. Simplify your life, use a BSD, where they cleaned up the mess as they did the cooking! Linux on 32 bit platforms is just sucky in comparison, and will get worse. And don’t even look at MIPS, where the padding is wrong.
5. Linux on MIPS
> Not a syscall, a whole implemntation of the Linux ABI. Unlike the lovely clean BSDs, Linux is different on each architecture, system calls randomly take arguments in different orders, and constants have different values, and there are special syscalls. But MIPS takes the biscuit, the whole packet of biscuits. It was made to be binary compatible with old SGI machines that don’t even exist, and has more syscall ABIs than I have had hot dinners. Clean it up! Make a new sane MIPS ABI and deprecate the old ones, nothing like adding another variant. So annoying I think I threw out all my MIPS machines, each different.
6. inotify, fanotify and friends
> Linux has no fewer than three file system change notification protocols. The first, dnotify hopped on ioctl‘s sidekick fcntl, while the two later ones, inotify and fanotify added a bunch more syscalls. You can use any of them, and they still will not provide the notification API you want for most applications. Most people use the second one, inotify and curse it. Did you know kqueue can do this on the BSDs?
7. personality
> Oozing in personality, but we just don’t get along. Basically obsolete, as the kernel can decide what kind of system emulation to do from binaries directly, it stays around with some use cases in persuading ./configure it is running on a 32 bit system. But it can turn off ASLR, and let the CVEs right into your system. We need less persoanlity!
8. gettimeofday
> Still has an obsolete timezone value from an old times when people thought timezones should go all the way to the kernel. Now we know that your computer should not know. Set its clock to UTC. Do the timezones in the UI based on where the user is, not the computer. You should use clock_gettime now. Don’t even talk to me about locales. This syscall is fast though, don’t use it for benchmarking, its in the VDSO.
9. splice and tee
> These, back in 2005 were a quite nice idea, although Linux said then “it is incomplete, the interfaces are ugly, and it will oops the system if anything goes wrong”. It won’t oops your system now, but usage has not taken off. The nice idea from Linus was that a pipe is just a ring buffer in the kernel, that can have a more general API and use cases for performant code, but a decade on it hasn’t really worked out. It was also supposed to be a more general sendfile, which in many ways was the successor of that Tux web server, but I think sendfile is still more widely used.
10. userfaultfd
> Yes, I like file descriptors. Yes CRIU is kind of cool. But userspace handling page faults? Is nothing sacred? I get that you can do this badly with a SIGSEGV handler, but talk about lipstick on a pig.
***
###OpenBSD 6.0 on an iMac G3 from 1999 (http://www.increasinglyadequate.com/macppc.html)
> A while ago I spent $50 for an iMac G3 (aka the iMac,1). This iconic model restored Apple's fortunes in the late '90s. Since the iMac G3 can still boot Mac OSes 8 and 9, I mostly use the machine to indulge a nostalgia for childhood schooldays spent poking at the operating system and playing Escape Velocity. But before I got around to that, I decided to try out the software that the previous owner had left on the machine. The antiquated OSX 10.2 install and 12 year old versions of Safari and Internet Explorer were too slow and old to use for anything. Updating to newer software was almost impossible; a later OSX is required to run the little PowerPC-compatible software still languishing in forgotten corners of the Internet. This got me thinking: could this machine be used, really used, nowadays? Lacking a newer OSX disc, I decided to try the most recent OpenBSD release. (And, since then, to re-try with each new OpenBSD release.) Below are the results of this experiment (plus a working xorg.conf file) and a few background notes.
Background
> This iMac is a Revision D iMac G3 in grape. It's part of the iMac,1 family of computers. This family includes all tray-loading iMac G3s. (Later iMac G3s had a slot-loading CD drive and different components.) Save for a slightly faster processor, a dedicated graphics card, and cosmetic tweaks to the case, my iMac is identical to the prior year's line-launching Bondi Blue iMac. My machine has had its memory upgraded from 32 MB to 320 MB. Thank Goodness.
> The Revision D iMac G3 shipped with Mac OS 8.5. It can run up to Mac OS 9.2.2 or OSX 10.3.9. Other operating systems that tout support for the iMac,1 include NetBSD, OpenBSD, and a shrinking number of Linux distributions.
> OpenBSD is simple (by design) and well-maintained. In contrast, NetBSD seems rather more complex and featureful, and I have heard grumbling that despite its reputation for portability, NetBSD really only works well on amd64. I'd test that assertion if OpenBSD's macppc installation instructions didn't seem much simpler than NetBSD's. Linux is even more complicated, although most distros are put together in a way that you can mostly ignore that complexity (until you can't). In the end I went with OpenBSD because I am familiar with it and because I like it.
Installing OpenBSD on the iMac,1
> Installing OpenBSD on this iMac was simple. It's the same procedure as installing OpenBSD on an amd64 rig. You put in the installation disc; you tell the machine to boot from it; and then you answer a few prompts, most of which simply ask you to press enter. In this case, OpenBSD recognizes all machine's hardware just fine, including sound and networking, though I had a little trouble with video.
> The OpenBSD documentation says video should just work and that an xorg.conf file isn't necessary. As such, it no longer ships with an xorg.conf file. Though that's never posed a problem on my other OpenBSD machines, it does here. Video doesn't work out of the box on my iMac,1. startx just blanks the screen. Fortunately, because the BSDs use a centralized development model where each operating system is stored in one repository, OpenBSD's website provides a web interface to the source code going back to the early days. I was able to find the last version of the sample xorg.conf that used to ship on macppc. With a little tweaking, I transformed that file into this one (https://www.increasinglyadequate.com/files/xorg.conf), with which video works just fine. Just drop it into your iMac's /etc/X11 directory. You'll also need to remember to set the machdep.allowaperture sysctl to 2 (e.g., as root run sysctl machdep.allowaperture=2), although the installer will do that automatically if you answer yes to the question about whether you plan to run X.
> All that being said, video performance is pretty poor. I am either doing something wrong, or OpenBSD doesn't have accelerated video for this iMac, or this machine is just really old! I will discuss performance below.
Running OpenBSD on the iMac,1
> The machine performs okay under OpenBSD. You can expect to ably run minimalistic software under minimalistic window managers. I tried dillo, mrxvt, and cmus under cwm and fvwm. Performance here was just fine. I also tried Firefox 26, 33, and 34 under fvwm and cwm. Firefox ran, but "modern," Javascript-heavy sites were an exercise in frustration; the 2015 version of CNN.com basically froze Firefox for 30 seconds or more. A lighter browser like dillo is doable.
> You'll notice that I used the past-tense to talk about Firefox. Firefox currently doesn't build on PowerPC on OpenBSD. Neither does Chromium. Neither do a fair number of applications. But whatever -- there's still a lot of lighter applications available, and it's these you'll use day-to-day on a decades-old machine.
> Lightweight window managers work okay, as you'd expect. You can even run heavier desktop environments, such as xfce, though you'll give up a lot of performance.
> I ran the Ubench benchmark on this iMac and two more modern machines also running OpenBSD. The benchmark seems like an old one; I don't know how (if at all) it accounts for hardware changes in the past 13 years. That is, I don't know if the difference in score accurately measures the difference in real-world performance. Here are the results anyway:
Conclusion
> Except for when I check to see if OpenBSD still works, I run Mac OS9 on this rig. I have faster and better machines for running OpenBSD. If I didn't -- if this rig were, improbably, all I had left, and I was waiting on the rush delivery of something modern -- then I would use OpenBSD on my iMac,1. I'd have to stick to lightweight applications, but at least they'd be up-to-date and running on a simple, stable, OS.
***
##News Roundup
###34th Chaos Communication Congress Schedule (https://events.ccc.de/congress/2017/Fahrplan/index.html)
Many talks are streamed live (http://streaming.media.ccc.de/34c3), a good mixture of english and german talks
May contain DTraces of FreeBSD (https://events.ccc.de/congress/2017/Fahrplan/events/9196.html)
Are all BSDs created equally? (https://events.ccc.de/congress/2017/Fahrplan/events/8968.html)
library operating systems (https://events.ccc.de/congress/2017/Fahrplan/events/8949.html)
Hardening Open Source Development (https://events.ccc.de/congress/2017/Fahrplan/events/9249.html)
***
###OpenBSD 6.2 + CDE (https://jamesdeagle.blogspot.co.uk/2017/12/openbsd-62-cde.html)
> If you've noticed a disruption in the time-space continuum recently, it is likely because I have finally been able to compile and install the Common Desktop Environment (CDE) in a current and actively-developed operating system (OpenBSD 6.2 in this case).
> This comes after so many attempts (across multiple platforms) that ended up with the build process prematurely stopping itself in its own tracks for a variety of infinitesimal reasons that were beyond my comprehension as a non-programmer, or when there was success it was not without some broken parts. As for the latter, I've been able to build CDE on OpenIndiana Hipster, but with an end product where I'm unable to change the color scheme in dtstyle (because "useColorObj" is set to "False"), with a default color scheme that is low-res and unpleasant. As for changing "useColorObj" to "True", I tried every recommended trick I could find online, but nothing worked.
> My recent attempts at installing CDE on OpenBSD (version 6.1) saw the process stop due to a number of errors that are pure gibberish to these naive eyes. While disappointing, it was par for the course within my miserable experience with trying to build this particular desktop environment. As I wrote in this space in November 2015, in the course of explaining part of my imperitive for installing Solaris 10:
> And so I have come to think of building the recently open-sourced CDE as being akin to a coffee mug I saw many years ago. One side of the mug read "Turn the mug to see how to keep an idiot busy." On the other side, it read "Turn the mug to see how to keep an idiot busy." I'm through feeling like an idiot, which is partially why I'm on this one-week journey with Solaris 10.
> While I thoroughly enjoyed running Solaris 10 on my ThinkPad T61p, and felt a devilish thrill at using it out in the open at my local MacBook- and iPhone-infested Starbucks and causing general befuddlement and consternation among the occasional prying yoga mom, I never felt like I could do much with it beyond explore the SunOS 5.10 command line and watch YouTube videos. While still supported by its current corporate owner (whose name I don't even want to type), it is no longer actively developed and is thus little more than a retro toy. I hated the idea of installing anything else over it, but productivity beckoned and it was time to tearfully and reluctantly drag myself off the dance floor.
> In any case, just last week I noticed that the Sourceforge page for the OpenBSD build had some 6.2-specific notes by way of a series of four patches, and so I decided 'what the heck, let's give this puppy another whirl'. After an initial abortive attempt at a build, I surmised that I hadn't applied the four patches correctly. A day or two later, I took a deep breath and tried again, this time resolving to not proceed with the time make World build command until I could see some sign of a successful patch process. (This time around, I downloaded the patches and moved them into the directory containing the CDE makefiles, and issued each patch command as patch | |||
| 14 Nov 2024 | 585: Infrastructure Administration Workstation | 00:49:06 | |
From Proxmox to FreeBSD - Story of a Migration, FreeBSD At 30: The History And Future Of The Most Popular BSD-Based OS, Using a dedicated administration workstation for my infrastructure, LibreSSL 4.0.0 Released, Plasma6 and FreeBSD 14, Replace gnu diff, diff3, and sdiff with BSD versions, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
From Proxmox to FreeBSD - Story of a Migration (https://it-notes.dragas.net/2024/10/21/from-proxmox-to-freebsd-story-of-a-migration/)
FreeBSD At 30: The History And Future Of The Most Popular BSD-Based OS (https://hackaday.com/2024/10/28/freebsd-at-30-the-history-and-future-of-the-most-popular-bsd-based-os/)
News Roundup
Using a dedicated administration workstation for my infrastructure (https://dataswamp.org/~solene/2024-10-19-my-admin-workstation.html)
LibreSSL 4.0.0 Released (https://www.undeadly.org/cgi?action=article;sid=20241015084629)
Plasma6 and FreeBSD 14 (https://euroquis.nl//kde/2024/10/08/freebsd14.html)
git: world - Replace gnu diff, diff3, and sdiff with BSD versions (https://lists.dragonflybsd.org/pipermail/commits/2024-October/923274.html)
Beastie Bits
- How to Upgrade FreeBSD KDE 5 to KDE 6 (https://www.youtube.com/watch?v=2OZtnqK3iMU)
***
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 17 Sep 2014 | 55: The Promised WLAN | 01:19:20 | |
Coming up this week, we'll be talking with Adrian Chadd about all things wireless, his experience with FreeBSD on various laptop hardware and a whole lot more. As usual, we've got the latest news and answers to all your emails, on BSD Now - the place to B.. SD.
This episode was brought to you by
Headlines
FreeBSD 10.1-BETA1 is out (http://ftp.freebsd.org/pub/FreeBSD/releases/ISO-IMAGES/10.1/)
The first maintenance update in the 10.x series of FreeBSD is on its way
Since we can't see a changelog yet, the 10-STABLE release notes (https://www.freebsd.org/relnotes/10-STABLE/relnotes/article.html) offer a glimpse at some of the new features and fixes that will be included in 10.1
The vt driver was merged from -CURRENT, lots of drivers were updated, lots of bugs were fixed and bhyve also got many improvements from 11
Initial UEFI support, multithreaded softupdates for UFS and many more things were added
You can check the release schedule (https://www.freebsd.org/releases/10.1R/schedule.html) for the planned release dates
Details for the various forms of release media can be found in the announcement (https://lists.freebsd.org/pipermail/freebsd-stable/2014-September/080106.html)
***
Remote headless OpenBSD installation (https://jcs.org/notaweblog/2014/09/12/remotely_installing_openbsd_on_a/)
A lot of server providers only offer a limited number of operating systems to be easily installed on their boxes
Sometimes you'll get lucky and they'll offer FreeBSD, but it's much harder to find ones that natively support other BSDs
This article shows how you can use a Linux-based rescue system, a RAM disk and QEMU to install OpenBSD on the bare metal of a server, headlessly and remotely
It required a few specific steps you'll want to take note of, but is extremely useful for those pesky hosting providers
***
Building a firewall appliance with pfSense (http://www.get-virtual.net/2014/09/16/build-firewall-appliance/)
In this article, we learn how to easily set up a gateway and wireless access point with pfSense on a Netgate ALIX2C3 APU (http://pcengines.ch/alix2c3.htm)
After the author's modem died, he decided to look into a more do-it-yourself option with pf and a tiny router board
The hardware he used has gigabit ports and a BSD-compatible wireless card, as well as enough CPU power for a modest workload and a few services (OpenVPN, etc.)
There's a lot of great pictures of the hardware and detailed screenshots, definitely worth a look
***
Receive Side Scaling - UDP testing (http://adrianchadd.blogspot.com/2014/09/receive-side-scaling-testing-udp.html)
Adrian Chadd has been working on RSS (Receive Side Scaling) in FreeBSD, and gives an update on the progress
He's using some quad core boxes with 10 gigabit ethernet for the tests
The post gives lots of stats and results from his network benchmark, as well as some interesting workarounds he had to do
He also provides some system configuration options, sysctl knobs, etc. (if you want to try it out)
And speaking of Adrian Chadd...
***
Interview - Adrian Chadd - adrian@freebsd.org (mailto:adrian@freebsd.org) / @erikarn (https://twitter.com/erikarn)
BSD on laptops, wifi, drivers, various topics
News Roundup
Sendmail removed from OpenBSD (http://undeadly.org/cgi?action=article&sid=20140916084251)
Mail server admins around the world are rejoicing (https://news.ycombinator.com/item?id=8324475), because sendmail is finally gone (http://marc.info/?l=openbsd-cvs&m=141081997917153&w=2) from OpenBSD
With OpenSMTPD being a part of the base system, sendmail became largely redundant and unneeded
If you've ever compared a "sendmail.cf" file to an "smtpd.conf" file... the different is as clear as night and day
5.6 will serve as a transitional release, including both sendmail and OpenSMTPD, but 5.7 will be the first release without it
If you still need it for some reason, sendmail will live in ports from now on
Hopefully FreeBSD will follow suit sometime in the future as well, possibly including DragonFly's mail transfer agent in base (instead of an entire mail server)
***
pfSense backups with pfmb (https://github.com/zinkwazi/pfmb)
We've mentioned the need for a tool to back up pfSense configs a number of times on the show
This script, hosted on github, does pretty much exactly that
It can connect to one (or more!) pfSense installations and back up the configuration
You can roll back or replace failed hardware very easily with its restore function
Everything is done over SSH, so it should be pretty secure
***
The Design and Implementation of the FreeBSD Operating System (http://www.amazon.com/dp/0321968972/)
We mentioned when the pre orders were up, but now "The Design and Implementation of the FreeBSD Operating System, 2nd edition" seems to be shipping out
If you're interested in FreeBSD development, or learning about the operating system internals, this is a great book to buy
We've even had all (http://www.bsdnow.tv/episodes/2013-10-02_stacks_of_cache) three (http://www.bsdnow.tv/episodes/2014_01_29-journaled_news_updates) authors (http://www.bsdnow.tv/episodes/2014_08_13-vpn_my_dear_watson) on the show before!
***
OpenBSD's systemd replacement updates (http://undeadly.org/cgi?action=article&sid=20140915064856)
We mentioned last week that the news of OpenBSD creating systemd wrappers was getting mainstream attention
One of the developers writes in to Undeadly, detailing what's going on and what the overall status is
He also clears up any confusion about "porting systemd to BSD" (that's not what's going on) or his code ever ending up in base (it won't)
The top comment as of right now is a Linux user asking if his systemd wrappers can be ported back to Linux... poor guy
***
Feedback/Questions
Brad writes in (http://slexy.org/view/s20jrx0nIf)
Ben writes in (http://slexy.org/view/s21hFUJ2ju)
Mathieu writes in (http://slexy.org/view/s21RgSzOv4)
Steve writes in (http://slexy.org/view/s2P1mzalPh)
***
| |||
| 02 Nov 2016 | 166: Pass that UNIX Pipe | 00:55:16 | |
This week on the show, we’re loaded up with great stories ranging from System call fuzzing, a history of UNIX Pipes, speeding up MySQL imports and more. Stay tuned, BSDNow is coming your way right now.
This episode was brought to you by
Headlines
System call fuzzing of OpenBSD amd64 using TriforceAFL (i.e. AFL and QEMU) (https://github.com/nccgroup/TriforceOpenBSDFuzzer)
The NCCGroup did a series of fuzz testing against the OpenBSD syscall interface, during which they found a number of vulnerabilities, we covered this back in the early summer
What we didn’t notice, is that they also made the tools they used available.
A combination of AFL (American Fuzzy Lop), QEMU, OpenBSD’s FlashRD image generation tool, and the “Triforce” driver
The other requirement is “a Linux box as host to run the fuzzer (other fuzzer hosts may work as well, we've only run TriforceAFL from a Linux host, specifically Debian/Ubuntu”
It would be interesting to see if someone could get this to run from a BSD host
It would also be interesting to run the same tests against the other BSDs
***
On the Early History and Impact of Unix: the Introduction of Pipes (http://people.fas.harvard.edu/~lib113/reference/unix/unix2.html)
Pipes are something we just take for granted today, but there was a time before pipes (How did anything get done?)
Ronda Hauben writes up a great look back at the beginning of UNIX, and specifically at how pipes were born:
One of the important developments in Unix was the introduction of pipes. Pipes had been suggested by McIlroy during the early days of creating Unix. Ritchie explains how "the idea, explained one afternoon on a blackboard, intrigued us but failed to ignite any immediate action. There were several objections to the idea as put....What a failure of imagination," he admits.(35) McIlroy concurs, describing how the initial effort to add pipes to Unix occurred about the same time in 1969 that Ritchie, Thompson and Canaday were outlining ideas for a file system. "That was when," he writes, "the simple pipeline as a way to combine programs, with data notationally propagating along a chain of (not necessarily concurrent) filters was articulated."(36) However, pipes weren't implemented in Unix until 1972.
We also have a great quote from McIlroy on the day pipes were first introduced:
Open Systems! Our Systems! How well those who were there remember the pipe-festooned garret where Unix took form. The excitement of creation drew people to work there amidst the whine of the computer's cool- ing fans, even though almost the same computer ac- cess, could be had from one's office or from home. Those raw quarters saw a procession of memorable events. The advent of software pipes precipitated a day-long orgy of one-liners....As people reveled in the power of functional composition in the large, which is even today unavailable to users of other systems.
The paper goes on to talk about the invention of other important tools, such as “grep”, “diff” and more. Well worth your time if you want a glimpse into the history of UNIX
***
Speeding up MySQL Import on FreeBSD (https://blog.feld.me/posts/2016/09/speeding-up-mysql-import-on-freebsd/)
Mark Felder writes a blog post explaining how to speed up MySQL bulk data imports
“I was recently tasked with rebuilding a readonly slave database server which only slaves a couple of the available databases. The backup/dump is straightforward and fast, but the restore was being excruciatingly slow. I didn't want to wait a week for this thing to finish, so I had to compile a list of optimizations that would speed up the process. This is the best way to do it on FreeBSD, assuming you're working with InnoDB. Additional optimizations may be required if you're using a different database engine.”
“Please note this is assuming no other databases are running on this MySQL instance. Some of these are rather dangerous and you wouldn't want to put other live data at risk.”
Most of the changes are meant to be temporary, used on a new server to import a dump of the database, then the settings are to be turned off.
Specifically:
sync_binlog = 0
innodbflushlogattrx_commit = 0
innodb-doublewrite = 0
He also prepends the following but of SQL before importing the data:
set sqllogbin=0; set autocommit=0; set uniquechecks=0; set foreignkey_checks=0;
You can also help yourself if your MySQL database lives on ZFS
zfs set recordsize=16k pool/var/db/mysql
zfs set redundant_metadata=most pool/var/db/mysql
Remember, this tuning is ONLY for the initial import, leaving these settings on long term risks losing 5-10 seconds of your data if the server reboots unexpectedly
zfs set sync=disabled pool/var/db/mysql
zfs set logbias=throughput pool/var/db/mysql
***
PostgreSQL and FreeBSD Quick Start (https://cwharton.com/blog/2016/10/postgresql-and-freebsd-quick-start/)
There’s lots of databases to choose from, but Postgres always has a special place on FreeBSD. Today we have a look at a ‘getting started’ guide for those taking the plunge and using it for the first time.
Naturally getting started will look familiar to many, a couple simple “pkg” and “sysrc” commands later, and you’ll be set.
After starting the service (With the “service” command) you’ll be ready to start setting up your postgres instance.
Next up you’ll need to create your initial user/password combo, and a database with access granted to this particular user.
If you plan to enable remote access to this DB server, you’ll need to make some adjustments to one of the .conf files, allowing other IP’s to connect. (If you are hosting something on the same system, this may not be needed)
Now yous should be good to go! Enjoy using your brand new Postgres database. If this is your first rodeo, maybe start with something easy, like Apache or Nginx + Wordpress to try it out.
***
News Roundup
OpenBSD vmm hypervisor test drive (https://www.youtube.com/watch?v=KE_7E1pXy5c)
As we asked for a week or two ago, someone has taken OpenBSD’s vmm for a test drive, and made a video of it
The command line interface for vmm, vmctl, looks quite easy to use. It takes an approach much closer to some of the bhyve management frameworks, rather than bhyve’s rather confusing set of switches
It also has a config file, the format of which looks very similar to what I designed for bhyveucl, and my first effort to integrate a config file into bhyve itself.
The video also looks at accessing the console, configuring the networking, and doing an OpenBSD install in a fresh VM
Currently vmm only supports running OpenBSD VMs
***
FreeBSD Foundation October 2016 Update (https://www.freebsdfoundation.org/wp-content/uploads/2016/10/FreeBSD-Foundation-October-2016-Update.pdf)
Wow, November is already upon us with the Holidays just around the corner. Before things get lost in the noise we wanted to highlight this update from the FreeBSD foundation.
Before getting into the stories, they helpfully provide a list of upcoming conferences for this fall/winter, which includes a couple of USENIX gatherings, and the Developer Summit / MeetBSD next week.
+The foundation gives us a quick hardware update initially, discussing some of the new ThunderX Cavium servers which are deployed (ARMv8 64Bit) and yes I’m drooling a bit. They also mention that work is ongoing for the RPi3 platform and PINE64.
GNN also has an article reprinted from the FreeBSD journal, talking about the achievement of making it to 11.0 over the span of 23 years now. Of course he mentions that the foundation is open to all, and welcomes donations to continue to keep up this tradition of good work being done.
Deb Goodkin gives us an update on the “Grace Hopper” convention that took place in Houston TX several weeks back. Roughly 14k women in Tech attended, which is a great turnout, and FreeBSD was well represented there.
Next we have a call to potential speakers, don’t forget that there are plenty of places you can help present about FreeBSD, not just at *BSD centered conferences, but the SCALES of the world as well.
We wrap up with a look at EuroBSDCon 2016, quite a nice writeup, again brought to us by Deb at the foundation, and includes a list of some of those recognized for their contributions to FreeBSD.
***
Adhokku – a toy PaaS powered by FreeBSD jails and Ansible (https://github.com/adhokku/adhokku)
Described as a toy Platform-as-a-Service, Adhokku is an ansible based automated jail creation framework
Based on the concept of Dokku, a single-host open source PaaS for Linux powered by Docker
When you deploy an application using Adhokku, Adhokku creates a new jail on the remote host and provisions it from a fixed clean state using the instructions in the Jailfile in your Git repository. All jails sit behind a reverse proxy that directs traffic to one of them based on the domain name or the IP address in the HTTP request. When a new jail has been provisioned for an application, Adhokku seamlessly reconfigures the reverse proxy to send traffic to it instead of the one currently active for that application.
The following instructions show how to get Adhokku and an example application running in a VM on your development machine using Vagrant. This process should require no FreeBSD-specific knowledge, through modifying the Jailfile to customize the application may.
This seems like an interesting project, and it is good to see people developing workflows so users familiar with docker etc, can easily use BSD instead
***
Installing OpenBSD 6.0 on your laptop is really hard (not) (http://sohcahtoa.org.uk/openbsd.html)
OpenBSD on a laptop? Difficult? Not hardly.
We have a great walkthrough by Keith Burnett, which demonstrates just how easy it can be to get up and running with an XFCE desktop from a fresh OpenBSD installation.
For those curious,this was all done with a Thinkpad X60 and 120GB SSD and OpenBSD 6.0.
He doesn’t really cover the install process itself, that is well covered by the link to the OpenBSD FAQ pages.
Once the system is up and running though, we start with the most important portion, getting working internet access (Via wifi)
Really just a few ‘ifconfig’ commands later and we are in business.
Step 2 was getting the package configuration going. (I’ve never understood why this is still a thing, but no fret, its easy enough to do)
With package repos available, now you can grab the binaries for XFCE and friends with just a few simple “pkg_add” commands
Steps 4-6 are some specific bits to enable XFCE services, and some handy things such as setting doas permissions to get USB mounting working (For graphical mount/unmount)
Lastly, keeping the system updated is important, so we have a nice tutorial on how to do that as well, using a handy “openup” script, which takes some of the guesswork out of it.
Bonus! Steps for doing FDE as also included, which isn’t for everyone, but you may want it
***
Beastie Bits
Pi-top with RPi-3 and FreeBSD HEAD (https://twitter.com/gvnn3/status/791475373380804608)
NetBSD 7.0.2 released (http://blog.netbsd.org/tnf/entry/netbsd_7_0_1_released1)
DragonflyBSD - git: kernel - Fix mmcsd read/write issues (http://lists.dragonflybsd.org/pipermail/commits/2016-October/624851.html)
A char device which implements an Enigma machine (FreeBSD & Linux) (https://github.com/rafael-santiago/dev-enigma)
***
Feedback/Questions
Matt - System Monitoring (http://pastebin.com/ayzvCuaq)
Tony - LLVM License (http://pastebin.com/r5axPSE7)
Ben - Thanks (http://pastebin.com/MNxCvUtX)
David - Write Cache (http://pastebin.com/RswFASqW)
Charles - Fonts (http://pastebin.com/e317a32f)
***
| |||
| 29 Aug 2019 | 313: In-Kernel TLS | 00:55:12 | |
OpenBSD on 7th gen Thinkpad X1 Carbon, how to install FreeBSD on a MacBook, Kernel portion of in-kernel TLS (KTLS), Boot Environments on DragonflyBSD, Project Trident Updates, vBSDcon schedule, and more.
Headlines
OpenBSD on the Thinkpad X1 Carbon 7th Gen (https://jcs.org/2019/08/14/x1c7)
Another year, another ThinkPad X1 Carbon, this time with a Dolby Atmos sound system and a smaller battery.
The seventh generation X1 Carbon isn't much different than the fifth and sixth generations. I opted for the non-vPro Core i5-8265U, 16Gb of RAM, a 512Gb NVMe SSD, and a matte non-touch WQHD display at ~300 nits. A brighter 500-nit 4k display is available, though early reports indicated it severely impacts battery life.
Gone are the microSD card slot on the back and 1mm of overall thickness (from 15.95mm to 14.95mm), but also 6Whr of battery (down to 51Whr) and a little bit of travel in the keyboard and TrackPoint buttons. I still very much like the feel of both of them, so kudos to Lenovo for not going too far down the Apple route of sacrificing performance and usability just for a thinner profile.
On my fifth generation X1 Carbon, I used a vinyl plotter to cut out stickers to cover the webcam, "X1 Carbon" branding from the bottom of the display, the power button LED, and the "ThinkPad" branding from the lower part of the keyboard deck.
See link for the rest of the article
How To Install FreeBSD On A MacBook 1,1 or 2,1 (http://lexploit.com/freebsdmacbook1-1-2-1/)
FreeBSD Setup For MacBook 1,1 and 2,1
FreeBSD with some additional setup can be installed on a MacBook 1,1 or 2,1. This article covers how to do so with FreeBSD 10-12.
Installing
FreeBSD can be installed as the only OS on your MacBook if desired. What you should have is:
A Mac OS X 10.4.6-10.7.5 installer. Unofficial versions modified for these MacBooks such as 10.8 also work.
A blank CD or DVD to burn the FreeBSD image to. Discs simply work best with these older MacBooks.
An ISO file of FreeBSD for x86. The AMD64 ISO does not boot due to the 32 bit EFI of these MacBooks.
Burn the ISO file to the blank CD or DVD. Once done, make sure it's in your MacBook and then power off the MacBook. Turn it on, and hold down the c key until the FreeBSD disc boots.
See link for the rest of the guide
News Roundup
Patch for review: Kernel portion of in-kernel TLS (KTLS) (https://svnweb.freebsd.org/base?view=revision&revision=351522)
One of the projects I have been working on for the past several months in conjunction with several other folks is upstreaming work from Netflix to handle some aspects of Transport Layer Security (TLS) in the kernel. In particular, this lets a web server use sendfile() to send static content on HTTPS connections. There is a lot more detail in the review itself, so I will spare pasting a big wall of text here. However, I have posted the patch to add the kernel-side of KTLS for review at the URL below. KTLS also requires other patches to OpenSSL and nginx, but this review is only for the kernel bits. Patches and reviews for the other bits will follow later.
https://reviews.freebsd.org/D21277
DragonFly Boot Enviroments (https://github.com/newnix/dfbeadm)
This is a tool inspired by the beadm utility for FreeBSD/Illumos systems that creates and manages ZFS boot environments. This utility in contrast is written from the ground up in C, this should provide better performance, integration, and extensibility than the POSIX sh and awk script it was inspired by. During the time this project has been worked on, beadm has been superseded by bectl on FreeBSD. After hammering out some of the outstanding internal logic issues, I might look at providing a similar interface to the command as bectl.
See link for the rest of the details
Project Trident Updates
19.08 Available (https://project-trident.org/post/2019-08-15_19.08_available/)
This is a general package update to the CURRENT release repository based upon TrueOS 19.08.
Legacy boot ISO functional again
This update includes the FreeBSD fixes for the “vesa” graphics driver for legacy-boot systems. The system can once again be installed on legacy-boot systems.
PACKAGE CHANGES FROM 19.07-U1
New Packages: 154
Deleted Packages: 394
Updated Packages: 4926
12-U3 Available (https://project-trident.org/post/2019-08-22_stable12-u3_available/)
This is the third general package update to the STABLE release repository based upon TrueOS 12-Stable.
PACKAGE CHANGES FROM STABLE 12-U2
New Packages: 105
Deleted Packages: 386
Updated Packages: 1046
vBSDcon (https://www.vbsdcon.com/schedule/)
vBSDcon 2019 will return to the Hyatt Regency in Reston, VA on September 5-7 2019.
***
Beastie Bits
The next NYCBUG meeting will be Sept 4 @ 18:45 (https://www.nycbug.org/index?action=view&id=10671)
Feedback/Questions
Tom - Questions (http://dpaste.com/1AXXK7G#wrap)
Michael - dfbeadm (http://dpaste.com/0PNEDYT#wrap)
Bostjan - Questions (http://dpaste.com/1N7T7BR#wrap)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 12 Mar 2024 | 550: Netware and Netmap | 00:53:16 | |
This week on the show, you're not too late to develop the future, netmap on czgbe, OpenZFS 2.2.3, SSH Brute Forcing, some unknown OpenBSD Features, Release notes for the latest Omni OS, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
When the Power Macintosh ran NetWare (featuring Wormhole and Cyberpunk) (https://oldvcr.blogspot.com/2023/12/when-power-macintosh-ran-netware.html)
You are not too late (https://kk.org/thetechnium/you-are-not-late/)
News Roundup
netmap on cxgbe interfaces (https://adventurist.me/posts/00318)
OpenZFS 2.2.3 (https://github.com/openzfs/zfs/releases/tag/zfs-2.2.3)
A recent abrupt change in Internet SSH brute force attacks against us (https://utcc.utoronto.ca/~cks/space/blog/sysadmin/SSHBruteForceAttacksAbruptlyDown)
Some OpenBSD features that aren't widely known (https://dataswamp.org/~solene/2024-02-20-rarely-known-openbsd-features.html)
Release Notes for OmniOS v11 r151048 (https://github.com/omniosorg/omnios-build/blob/44731424e67c8aaafe5c4e500fe7c4544a22f0f6/doc/ReleaseNotes.md#r151048o-2024-02-15)
The Making of RP2040 Doom (https://kilograham.github.io/rp2040-doom/)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Brendan - Log Files (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/550/feedback/Brendan%20-%20Log%20Files.md)
Mischa - EuroBSDcon (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/550/feedback/Mischa%20-%20EuroBSDcon.md)
Sebastiano - Sed (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/550/feedback/Sebastiano%20-%20Sed.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 26 Mar 2020 | 343: FreeBSD, Corona: Fight! | 00:39:04 | |
Fighting the Coronavirus with FreeBSD, Wireguard VPN Howto in OPNsense, NomadBSD 1.3.1 available, fresh GhostBSD 20.02, New FuryBSD XFCE and KDE images, pf-badhost 0.3 released, and more.
Headlines
Fighting the Coronavirus with FreeBSD (https://www.leidinger.net/blog/2020/03/19/fighting-the-coronavirus-with-freebsd-foldinghome/)
Here is a quick HOWTO for those who want to provide some FreeBSD based compute resources to help finding vaccines.
UPDATE 2020-03-22: 0mp@ made a port out of this, it is in “biology/linux-foldingathome”.
Per default it will now pick up some SARS-CoV‑2 (COVID-19) related folding tasks. There are some more config options (e.g. how much of the system resources are used). Please refer to the official Folding@Home site for more information about that. Be also aware that there is a big rise in compute resources donated to Folding@Home, so the pool of available work units may be empty from time to time, but they are working on adding more work units. Be patient.
How to configure the Wireguard VPN in OPNsense (https://homenetworkguy.com/how-to/configure-wireguard-opnsense/)
WireGuard is a modern designed VPN that uses the latest cryptography for stronger security, is very lightweight, and is relatively easy to set up (mostly). I say ‘mostly’ because I found setting up WireGuard in OPNsense to be more difficult than I anticipated. The basic setup of the WireGuard VPN itself was as easy as the authors claim on their website, but I came across a few gotcha's. The gotcha's occur with functionality that is beyond the scope of the WireGuard protocol so I cannot fault them for that. My greatest struggle was configuring WireGuard to function similarly to my OpenVPN server. I want the ability to connect remotely to my home network from my iPhone or iPad, tunnel all traffic through the VPN, have access to certain devices and services on my network, and have the VPN devices use my home's Internet connection.
WireGuard behaves more like a SSH server than a typical VPN server. With WireGuard, devices which have shared their cryptographic keys with each other are able to connect via an encrypted tunnel (like a SSH server configured to use keys instead of passwords). The devices that are connecting to one another are referred to as “peer” devices. When the peer device is an OPNsense router with WireGuard installed, for instance, it can be configured to allow access to various resources on your network. It becomes a tunnel into your network similar to OpenVPN (with the appropriate firewall rules enabled). I will refer to the WireGuard installation on OPNsense as the server rather than a “peer” to make it more clear which device I am configuring unless I am describing the user interface because that is the terminology used interchangeably by WireGuard.
The documentation I found on WireGuard in OPNsense is straightforward and relatively easy to understand, but I had to wrestle with it for a little while to gain a better understanding on how it should be configured. I believe it was partially due to differing end goals – I was trying to achieve something a little different than the authors of other wiki/blog/forum posts. Piecing together various sources of information, I finally ended up with a configuration that met the goals stated above.
News Roundup
NomadBSD 1.3.1 (https://nomadbsd.org/index.html#1.3.1)
NomadBSD 1.3.1 has recently been made available. NomadBSD is a lightweight and portable FreeBSD distribution, designed to run on live on a USB flash drive, allowing you to plug, test, and play on different hardware. They have also started a forum as of yesterday, where you can ask questions and mingle with the NomadBSD community. Notable changes in 1.3.1 are base system upgraded to FreeBSD 12.1-p2. automatic network interface setup improved, image size increased to over 4GB, Thunderbird, Zeroconf, and some more listed below.
GhostBSD 20.02 (https://ghostbsd.org/20.02_release_announcement)
Eric Turgeon, main developer of GhostBSD, has announced version 20.02 of the FreeBSD based operating system. Notable changes are ZFS partition into the custom partition editor installer, allowing you to install alongside with Windows, Linux, or macOS. Other changes are force upgrade all packages on system upgrade, improved update station, and powerd by default for laptop battery performance.
New FuryBSD XFCE and KDE images (https://www.furybsd.org/new-furybsd-12-1-based-images-are-available-for-xfce-and-kde/)
This new release is now based on FreeBSD 12.1 with the latest FreeBSD quarterly packages. This brings XFCE up to 4.14, and KDE up to 5.17. In addition to updates this new ISO mostly addresses community bugs, community enhancement requests, and community pull requests. Due to the overwhelming amount of reports with GitHub hosting all new releases are now being pushed to SourceForge only for the time being. Previous releases will still be kept for archive purposes.
pf-badhost 0.3 Released (https://www.geoghegan.ca/pfbadhost.html)
pf-badhost is a simple, easy to use badhost blocker that uses the power of the pf firewall to block many of the internet's biggest irritants. Annoyances such as SSH and SMTP bruteforcers are largely eliminated. Shodan scans and bots looking for webservers to abuse are stopped dead in their tracks. When used to filter outbound traffic, pf-badhost blocks many seedy, spooky malware containing and/or compromised webhosts.
Beastie Bits
DragonFly i915 drm update (https://www.dragonflydigest.com/2020/03/23/24324.html)
CShell is punk rock (http://blog.snailtext.com/posts/cshell-is-punk-rock.html)
The most surprising Unix programs (https://minnie.tuhs.org/pipermail/tuhs/2020-March/020664.html)
Feedback/Questions
Master One - Torn between OpenBSD and FreeBSD (http://dpaste.com/102HKF5#wrap)
Brad - Follow up to Linus ZFS story (http://dpaste.com/1VXQA2Y#wrap)
Filipe Carvalho - Call for Portuguese BSD User Groups (http://dpaste.com/2H7S8YP)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 18 May 2023 | 507: Michael W. Lucas Interview | 00:58:41 | |
Author Michael W. Lucas joins us in this interview to talk about his latest book projects. Find out what he’s up to regarding mail servers, conferences, his views on ChatGPT, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Interview - Michael W. Lucas - mwl@mwl.io (mailto:mwl@mwl.io)
OpenBSD Mastery Filesystems
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
- Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Special Guest: Michael W Lucas.
| |||
| 05 Dec 2019 | 327: ZFS Rename Repo | 01:23:27 | |
We read FreeBSD’s third quarterly status report, OpenBSD on Sparc64, ZoL repo move to OpenZFS, GEOM NOP, keeping NetBSD up-to-date, and more.
Headlines
FreeBSD third quarterly status report for 2019 (https://www.freebsd.org/news/status/report-2019-07-2019-09.html)
This quarter the reports team has been more active than usual thanks to a better organization: calls for reports and reminders have been sent regularly, reports have been reviewed and merged quickly (I would like to thank debdrup@ in particular for his reviewing work).
Efficiency could still be improved with the help of our community. In particular, the quarterly team has found that many reports have arrived in the last days before the deadline or even after. I would like to invite the community to follow the guidelines below that can help us sending out the reports sooner.
Starting from next quarter, all quarterly status reports will be prepared the last month of the quarter itself, instead of the first month after the quarter's end. This means that deadlines for submitting reports will be the 1st of January, April, July and October.
Next quarter will then be a short one, covering the months of November and December only and the report will probably be out in mid January.
OpenBSD on Sparc64 (https://eerielinux.wordpress.com/2019/10/10/openbsd-on-sparc64-6-0-to-6-5/)
OpenBSD, huh? Yes, I usually write about FreeBSD and that’s in fact what I tried installing on the machine first. But I ran into problems with it very early on (never even reached single user mode) and put it aside for later. Since I powered up the SunFire again last month, I needed an OS now and chose OpenBSD for the simple reason that I have it available.
First I wanted to call this article simply “OpenBSD on SPARC” – but that would have been misleading since OpenBSD used to support 32-bit SPARC processors, too. The platform was just put to rest after the 5.9 release.
Version 6.0 was the last release of OpenBSD that came on CD-ROM. When I bought it, I thought that I’d never use the SPARC CD. But here was the chance! While it is an obsolete release, it comes with the cryptographic signatures to verify the next release. So the plan is to start at 6.0 as I can trust the original CDs and then update to the latest release. This will also be an opportunity to recap on some of the things that changed over the various versions.
News Roundup
ZoL repo move to OpenZFS (https://zfsonlinux.topicbox.com/groups/zfs-discuss/T13eedc32607dab41/zol-repo-move-to-openzfs)
Because it will contain the ZFS source code for both Linux and FreeBSD, we will rename the "ZFSonLinux" code repository to "OpenZFS". Specifically, the repo at http://github.com/ZFSonLinux/zfs will be moved to the OpenZFS organization, at http://github.com/OpenZFS/zfs.
The next major release of ZFS for Linux and FreeBSD will be "OpenZFS 2.0", and is expected to ship in 2020.
Mcclure111 Sun Thread (https://twitter.com/mcclure111/status/1196557401710837762)
A long time ago— like 15 years ago— I worked at Sun Microsystems. The company was nearly dead at the time (it died a couple years later) because they didn't make anything that anyone wanted to buy anymore. So they had a lot of strange ideas about how they'd make their comeback.
GEOM NOP (https://oshogbo.vexillium.org/blog/71/)
Sometimes while testing file systems or applications you want to simulate some errors on the disk level. The first time I heard about this need was from Baptiste Daroussin during his presentation at AsiaBSDCon 2016. He mentioned how they had built a test lab with it. The same need was recently discussed during the PGCon 2019, to test a PostgreSQL instance. If you are FreeBSD user, I have great news for you: there is a GEOM provider which allows you to simulate a failing device.
GNOP allows us to configure transparent providers from existing ones. The first interesting option of it is that we can slice the device into smaller pieces, thanks to the ‘offset option’ and ‘stripsesize’. This allows us to observe how the data on the disk is changing. Let’s assume that we want to observe the changes in the GPT table when the GPT flags are added or removed (for example the bootme flags which are described here). We can use dd every time and analyze it using absolute values from the disks.
Keeping NetBSD up-to-date with pkg_comp 2.0 (https://jmmv.dev/2017/02/pkg_comp-2.0-tutorial-netbsd.html)
This is a tutorial to guide you through the shiny new pkg_comp 2.0 on NetBSD.
Goals: to use pkg_comp 2.0 to build a binary repository of all the packages you are interested in; to keep the repository fresh on a daily basis; and to use that repository with pkgin to maintain your NetBSD system up-to-date and secure.
This tutorial is specifically targeted at NetBSD but should work on other platforms with some small changes. Expect, at the very least, a macOS-specific tutorial as soon as I create a pkg_comp standalone installer for that platform.
Beastie Bits
DragonFly - Radeon Improvements (http://lists.dragonflybsd.org/pipermail/commits/2019-November/720070.html)
NomadBSD review (https://www.youtube.com/watch?v=7DglP7SbnlA&feature=share)
Spongebob OpenBSD Security Comic (https://files.yukiisbo.red/openbsd_claim.png)
Forth : The Early Years (https://colorforth.github.io/HOPL.html)
LCM+L PDP-7 booting and running UNIX Version 0 (https://www.youtube.com/watch?v=pvaPaWyiuLA)
Feedback/Questions
Chris - Ctrl-T (http://dpaste.com/284E5BV)
Improved Ctrl+t that shows kernel backtrace (https://asciinema.org/a/xfSpvPT61Cnd9iRgbfIjT6kYj)
Brian - Migrating NexentaStore to FreeBSD/FreeNAS (http://dpaste.com/05GDK8H#wrap)
Avery - How to get involved (http://dpaste.com/26KW801#wrap)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 21 Mar 2018 | 238: VLAN-Zezes-ki in Hardware | 02:03:38 | |
Looking at Lumina Desktop 2.0, 2 months of KPTI development in SmartOS, OpenBSD email service, an interview with Ryan Zezeski, NomadBSD released, and John Carmack's programming retreat with OpenBSD.
This episode was brought to you by
Headlines
Looking at Lumina Desktop 2.0 (https://www.trueos.org/blog/looking-lumina-desktop-2-0/)
A few weeks ago I sat down with Lead Developer Ken Moore of the TrueOS Project to get answers to some of the most frequently asked questions about Lumina Desktop from the open source community. Here is what he said on Lumina Desktop 2.0. Do you have a question for Ken and the rest of the team over at the TrueOS Project? Make sure to read the interview and comment below. We are glad to answer your questions!
Ken: Lumina Desktop 2.0 is a significant overhaul compared to Lumina 1.x. Almost every single subsystem of the desktop has been streamlined, resulting in a nearly-total conversion in many important areas.
With Lumina Desktop 2.0 we will finally achieve our long-term goal of turning Lumina into a complete, end-to-end management system for the graphical session and removing all the current runtime dependencies from Lumina 1.x (Fluxbox, xscreensaver, compton/xcompmgr). The functionality from those utilities is now provided by Lumina Desktop itself.
Going along with the session management changes, we have compressed the entire desktop into a single, multi-threaded binary. This means that if any rogue script or tool starts trying to muck about with the memory used by the desktop (probably even more relevant now than when we started working on this), the entire desktop session will close/crash rather than allowing targeted application crashes to bypass the session security mechanisms. By the same token, this also prevents “man-in-the-middle” type of attacks because the desktop does not use any sort of external messaging system to communicate (looking at you dbus). This also gives a large performance boost to Lumina Desktop
The entire system for how a user’s settings get saved and loaded has been completely redone, making it a “layered” settings system which allows the default settings (Lumina) to get transparently replaced by system settings (OS/Distributor/SysAdmin) which can get replaced by individual user settings. This results in the actual changes in the user setting files to be kept to a minimum and allows for a smooth transition between updates to the OS or Desktop. This also provides the ability to “restrict” a user’s desktop session (based on a system config file) to the default system settings and read-only user sessions for certain business applications.
The entire graphical interface has been written in QML in order to fully-utilize hardware-based GPU acceleration with OpenGL while the backend logic and management systems are still written entirely in C++. This results in blazing fast performance on the backend systems (myriad multi-threaded C++ objects) as well as a smooth and responsive graphical interface with all the bells and whistles (drag and drop, compositing, shading, etc).
Q: Are there future plans to implement something like Lumina in a MAC Jail?
While I have never tried out Lumina in a MAC jail, I do not see anything on that page which should stop it from running in one right now. Lumina is already designed to be run as an unpriviledged user and is very smart about probing the system to find out what is/not available before showing anything to the user. The only thing that comes to mind is that you might need to open up some other system devices so that X11 itself can draw to the display (graphical environment setup is a bit different than CLI environment).
Q: I look forward to these changes. I know the last time I used it when I would scroll I would get flashes like the refresh rate was not high enough. It will be nice to have a fast system as well as I know with the more changes Linux is becoming slower. Not once it has loaded but in the loading process. I will do another download when these changes come out and install again and maybe stay this time.
If I recall correctly, one of the very first versions of Lumina (pre-1.0) would occasionally flicker. If that is still happening, you might want to verify that you are using the proper video driver for your hardware and/or enable the compositor within the Lumina settings.
Q: Why was enlightenment project not considered for TrueOS? It is BSD licensed and is written in C.
This was a common question about 4(?) years ago with the first release of the Lumina desktop and it basically boiled down to long-term support and reliability of the underlying toolkit. Some of the things we had to consider were: cross-platform/cross-architecture support, dependency reliability and support framework (Qt5 > EFL), and runtime requirements and dependency tracking (Qt5 is lighter than the EFL). That plus the fact that the EFL specifically states that it is linux-focused and the BSD’s are just an afterthought (especially at the time we were doing the evaluation).
Q: I have two questions.
1) The default layout of Unity(menu bar with actual menu entries on top and icon dock on the side) is one of the few things I liked about my first voyage into non-Windows systems, and have been missing since moving on to other distros(and now also other non-Linux systems). However in 1.4.0 screenshots on Lumina’s site, the OSX-like layout has the menu attached to the window. Will 2.0 be able to have the menus on the bar?
2) Is there any timeline for a public release, or are you taking a “when it’s ready” approach?
In Lumina you can already put panels on the left/right side of the screen and give you something like the layout of the Unity desktop. The embedded menu system is not available in Lumina because that is not a specification supported by X11 and the window manager standards at the present time. The way that functionality is currently run on Linux is a hacky-bypass of the display system which only really works with the GTK3 and Qt5 toolkits, resulting in very odd overall desktop behavior in mixed environments where some apps use other graphical toolkits.
We are targetting the 18.06 STABLE release of TrueOS for Lumina 2, but that is just a guideline and if necessary we will push back the release date to allow for additional testing/fixing as needed.
A long two months (https://blog.cooperi.net/a-long-two-months)
IllumOS/SmartOS developer Alex Wilson describes the journey of developing KPTI for IllumOS
> On Monday (January 1st) I had the day off work for New Year's day, as is usual in most of the western world, so I slept in late. Lou and her friend decided to go to the wax museum and see several tourist attractions around SF, and I decided to pass the day at home reading. That afternoon, work chat started talking about a Tumblr post by pythonsweetness about an Intel hardware security bug. At the time I definitely did not suspect that this was going to occupy most of my working life for the next (almost) two months.
Like many people who work on system security, I had read Anders Fogh's post about a "Negative Result" in speculative execution research in July of 2017. At the time I thought it was an interesting writeup and I remember being glad that researchers were looking into this area. I sent the post to Bryan and asked him about his thoughts on it at the time, to which he replied saying that "it would be shocking if they left a way to directly leak out memory in the speculative execution". None of us seriously thought that there would be low-hanging fruit down that research path, but we also felt it was important that there was someone doing work in the area who was committed to public disclosure.
At first, after reading the blog post on Monday, we thought (or hoped) that the bug might "just" be a KASLR bypass and wouldn't require a lot of urgency. We tried to reach out to Intel at work to get more information but were met with silence. (We wouldn't hear back from them until after the disclosure was already made public.) The speculation on Tuesday intensified, until finally on Wednesday morning I arrived at the office to find links to late Tuesday night tweets revealing exploits that allowed arbitrary kernel memory reads.
Wednesday was not a happy day. Intel finally responded to our emails -- after they had already initiated public disclosure. We all spent a lot of time reading. An arbitrary kernel memory read (an info leak) is not that uncommon as far as bugs go, but for the most part they tend to be fairly easy to fix. The thing that makes the Meltdown and Spectre bugs particularly notable is that in order to mitigate them, a large amount of change is required in very deep low-level parts of the kernel. The kind of deep parts of the kernel where there are 20-year old errata workarounds that were single-line changes that you have to be very careful to not accidentally undo; the kind of parts where, as they say, mortals fear to tread.
On Friday we saw the patches Matthew Dillon put together for DragonFlyBSD for the first time. These were the first patches for KPTI that were very straightforward to read and understand, and applied to a BSD-derived kernel that was similar to those I'm accustomed to working on.
To mitigate Meltdown (and partially one of the Spectre variants), you have to make sure that speculative execution cannot reach any sensitive data from a user context. This basically means that the pages the kernel uses for anything potentially sensitive have to be unmapped when we are running user code. Traditionally, CPUs that were built to run a multi-user, UNIX-like OS did this by default (SPARC is an example of such a CPU which has completely separate address spaces for the kernel and userland). However, x86 descends from a single-address-space microcontroller that has grown up avoiding backwards-incompatible changes, and has never really introduced a clean notion of multiple address spaces (segmentation is the closest feature really, and it was thrown out for 64-bit AMD64). Instead, operating systems for x86 have generally wound up (at least in the post-AMD64 era) with flat address space models where the kernel text and data is always present in the page table no matter whether you're in user or kernel mode. The kernel mappings simply have the "supervisor" bit set on them so that user code can't directly access them.
The mitigation is basically to stop doing this: to stop mapping the kernel text, data and other memory into the page table while we're running in userland. Unfortunately, the x86 design does not make this easy. In order to be able to take interrupts or traps, the CPU has to have a number of structures mapped in the current page table at all times. There is also no ability to tell an x86 CPU that you want it to switch page tables when an interrupt occurs. So, the code that we jump to when we take an interrupt, as well as space for a stack to push context onto have to be available in both page tables. And finally, of course, we need to be able to figure out somehow what the other page table we should switch to is when we enter the kernel.
When we looked at the patches for Linux (and also the DragonFlyBSD patches at the time) on Friday and started asking questions, it became pretty evident that the initial work done by both was done under time constraints. Both had left the full kernel text mapped in both page tables, and the Linux trampoline design seemed over-complex. I started talking over some ideas with Robert Mustacchi about ways to fix these and who we should talk to, and reached out to some of my old workmates from the University of Queensland who were involved with OpenBSD. It seemed to me that the OpenBSD developers would care about these issues even more than we did, and would want to work out how to do the mitigation right.
I ended up sending an email to Philip Guenther on Friday afternoon, and on Saturday morning I drove an hour or so to meet up with him for coffee to talk page tables and interrupt trampolines. We wound up spending a good 6 hours at the coffee shop, and I came back with several pages of notes and a half-decent idea of the shape of the work to come.
One detail we missed that day was the interaction of per-CPU structures with per-process page tables. Much of the interrupt trampoline work is most easily done by using per-CPU structures in memory (and you definitely want a per-CPU stack!). If you combine that with per-process page tables, however, you have a problem: if you leave all the per-CPU areas mapped in all the processes, you will leak information (via Meltdown) about the state of one process to a different one when taking interrupts. In particular, you will leak things like %rip, which ruins all the work being done with PIE and ASLR pretty quickly. So, there are two options: you can either allocate the per-CPU structures per-process (so you end up with $NCPUS * $NPROCS of them); or you can make the page tables per-CPU.
OpenBSD, like Linux and the other implementations so far, decided to go down the road of per-CPU per-process pages to solve this issue. For illumos, we took the other route.
In illumos, it turned out that we already had per-CPU page tables. Robert and I re-discovered this on the Sunday of that week. We use them for 32-bit processes due to having full P>V PAE support in our kernel (which is, as it turns out, relatively uncommon amongst open-source OS). The logic to deal with creating and managing them and updating them was all already written, and after reading the code we concluded we could basically make a few small changes and re-use all of it. So we did.
By the end of that second week, we had a prototype that could get to userland. But, when working on this kind of kernel change we have a rule of thumb we use: after the first 70% of the patch is done and we can boot again, now it's time for the second 70%. In fact it turned out to be more like the second 200% for us -- a tedious long tail of bugs to solve that ended up necessitating some changes in the design as well.
At first we borrowed the method that Matt Dillon used for DragonFlyBSD, by putting the temporary "stack" space and state data for the interrupt trampolines into an extra page tacked onto the end of *%gs (in illumos the structure that lives there is the cpu_t).
If you read the existing logic in interrupt handlers for dealing with %gs though, you will quickly notice that the corner cases start to build up. There are a bunch of situations where the kernel temporarily alters %gs, and some of the ways to mess it up have security consequences that end up being worse than the bug we're trying to fix. As it turns out, there are no less than 3 different ways that ISRs use to try to get to having the right cpu_t in %gs on illumos, as it turns out, and they are all subtly different. Trying to tell which you should use when requires a bunch of test logic that in turn requires branches and changes to the CPU state, which is difficult to do in a trampoline where you're trying to avoid altering that state as much as possible until you've got the real stack online to push things into.
I kept in touch with Philip Guenther and Mike Larkin from the OpenBSD project throughout the weeks that followed. In one of the discussions we had, we talked about the NMI/MCE handlers and the fact that their handling currently on OpenBSD neglected some nasty corner-cases around interrupting an existing trap handler. A big part of the solution to those issues was to use a feature called IST, which allows you to unconditionally change stacks when you take an interrupt.
Traditionally, x86 only changes the stack pointer (%rsp on AMD64) while taking an interrupt when there is a privilege level change. If you take an interrupt while already in the kernel, the CPU does not change the stack pointer, and simply pushes the interrupt stack frame onto the stack you're already using. IST makes the change of stack pointer unconditional. If used unwisely, this is a bad idea: if you stay on that stack and turn interrupts back on, you could take another interrupt and clobber the frame you're already in. However, in it I saw a possible way to simplify the KPTI trampoline logic and avoid having to deal with %gs.
A few weeks into the project, John Levon joined us at work. He had previously worked on a bunch of Xen-related stuff as well as other parts of the kernel very close to where we were, so he quickly got up to speed with the KPTI work as well. He and I drafted out a "crazy idea" on the whiteboard one afternoon where we would use IST for all interrupts on the system, and put the "stack" they used in the KPTI page on the end of the cpu_t. Then, they could easily use stack-relative addresses to get the page table to change to, then pivot their stack to the real kernel stack memory, and throw away (almost) all the conditional logic. A few days later, we had convinced each other that this was the way to go.
Two of the most annoying x86 issues we had to work around were related to the SYSENTER instruction. This instruction is used to make "fast" system calls in 32-bit userland. It has a couple of unfortunate properties: firstly, it doesn't save or restore RFLAGS, so the kernel code has to take care of this (and be very careful not to clobber any of it before saving or after restoring it). Secondly, if you execute SYSENTER with the TF ("trap"/single-step flag) set by a debugger, the resulting debug trap's frame points at kernel code instead of the user code where it actually happened. The first one requires some careful gymnastics on the entry and return trampolines specifically for SYSENTER, while the second is a nasty case that is incidentally made easier by using IST. With IST, we can simply make the debug trap trampoline check for whether we took the trap in another trampoline's code, and reset %cr3 and the destination stack. This works for single-stepping into any of the handlers, not just the one for SYSENTER.
To make debugging easier, we decided that traps like the debug/single-step trap (as well as faults like page faults, #GP, etc.) would push their interrupt frame in a different part of the KPTI state page to normal interrupts. We applied this change to all the traps that can interrupt another trampoline (based on the instructions we used). These "paranoid" traps also set a flag in the KPTI struct to mark it busy (and jump to the double-fault handler if it is), to work around some bugs where double-faults are not correctly generated.
It's been a long and busy two months, with lots of time spent building, testing, and validating the code. We've run it on as many kinds of machines as we could get our hands on, to try to make sure we catch issues. The time we've spent on this has been validated several times in the process by finding bugs that could have been nasty in production.
One great example: our patches on Westmere-EP Xeons were causing busy machines to throw a lot of L0 I-cache parity errors. This seemed very mysterious at first, and it took us a few times seeing it to believe that it was actually our fault. This was actually caused by the accidental activation of a CPU errata for Westmere (B52, "Memory Aliasing of Code Pages May Cause Unpredictable System Behaviour") -- it turned out we had made a typo and put the "cacheable" flag into a variable named flags instead of attrs where it belonged when setting up the page tables. This was causing performance degradation on other machines, but on Westmere it causes cache parity errors as well. This is a great example of the surprising consequences that small mistakes in this kind of code can end up having. In the end, I'm glad that that erratum existed, otherwise it may have been a long time before we caught that bug.
As of this week, Mike and Philip have committed the OpenBSD patches for KPTI to their repository, and the patches for illumos are out for review. It's a nice kind of symmetry that the two projects who started on the work together after the public disclosure at the same time are both almost ready to ship at the same time at the other end. I'm feeling hopeful, and looking forward to further future collaborations like this with our cousins, the BSDs.
The IllumOS work has since landed, on March 12th (https://github.com/joyent/illumos-joyent/commit/d85fbfe15cf9925f83722b6d62da49d549af615c)
***
OpenBSD Email Service (https://github.com/vedetta-com/caesonia)
Features
Efficient: configured to run on min. 512MB RAM and 20GB SSD, a KVM (cloud) VPS for around $2.50/mo
15GB+ uncompressed Maildir, rivals top free-email providers (grow by upgrading SSD)
Email messages are gzip compressed, at least 1/3 more space with level 6 default
Server side full text search (headers and body) can be enabled (to use the extra space)
Mobile data friendly: IMAPS connections are compressed
Subaddress (+tag) support, to filter and monitor email addresses
Virtual domains, aliases, and credentials in files, Berkeley DB, or SQLite3
Naive Bayes rspamd filtering with supervised learning: the lowest false positive spam detection rates
Carefree automated Spam/ and Trash/ cleaning service (default: older than 30 days)
Automated quota management, gently assists when over quota
Easy backup MX setup: using the same configuration, install in minutes on a different host
Worry-free automated master/master replication with backup MX, prevents accidental loss of email messages
Resilient: the backup MX can be used as primary, even when the primary is not down, both perfect replicas
Flexible: switching roles is easy, making the process of changing VPS hosts a breeze (no downtime)
DMARC (with DKIM and SPF) email-validation system, to detect and prevent email spoofing
Daily (spartan) stats, to keep track of things
Your sieve scripts and managesieve configuration, let's get started
Considerations
By design, email message headers need to be public, for exchanges to happen. The body of the message can be encrypted by the user, if desired. Moreover, there is no way to prevent the host from having access to the virtual machine. Therefore, full disk encryption (at rest) may not be necessary.
Given our low memory requirements, and the single-purpose concept of email service, Roundcube or other web-based IMAP email clients should be on a different VPS.
Antivirus software users (usually) have the service running on their devices. ClamAV can easily be incorporated into this configuration, if affected by the types of malware it protects against, but will require around 1GB additional RAM (or another VPS).
Every email message is important, if properly delivered, for Bayes classification. At least 200 ham and 200 spam messages are required to learn what one considers junk. By default (change to use case), a rspamd score above 50% will send the message to Spam/. Moving messages in and out of Spam/ changes this score. After 95%, the message is flagged as "seen" and can be safely ignored.
Spamd is effective at greylisting and stopping high volume spam, if it becomes a problem. It will be an option when IPv6 is supported, along with bgp-spamd.
System mail is delivered to an alias mapped to a virtual user served by the service. This way, messages are guaranteed to be delivered via encrypted connection. It is not possible for real users to alias, nor mail an external mail address with the default configuration. e.g. puffy@mercury.example.com is wheel, with an alias mapped to (virtual) puffy@example.com, and user (puffy) can be different for each.
Interview - Ryan Zezeski - rpz@joyent.com (mailto:rpz@joyent.com) / @rzezeski (https://twitter.com/rzezeski)
News Roundup
John Carmack's programming retreat to hermit coding with OpenBSD (https://www.facebook.com/permalink.php?story_fbid=2110408722526967&id=100006735798590)
After a several year gap, I finally took another week-long programming retreat, where I could work in hermit mode, away from the normal press of work. My wife has been generously offering it to me the last few years, but I’m generally bad at taking vacations from work.
As a change of pace from my current Oculus work, I wanted to write some from-scratch-in-C++ neural network implementations, and I wanted to do it with a strictly base OpenBSD system. Someone remarked that is a pretty random pairing, but it worked out ok.
Despite not having actually used it, I have always been fond of the idea of OpenBSD — a relatively minimal and opinionated system with a cohesive vision and an emphasis on quality and craftsmanship. Linux is a lot of things, but cohesive isn’t one of them.
I’m not a Unix geek. I get around ok, but I am most comfortable developing in Visual Studio on Windows. I thought a week of full immersion work in the old school Unix style would be interesting, even if it meant working at a slower pace. It was sort of an adventure in retro computing — this was fvwm and vi. Not vim, actual BSD vi.
In the end, I didn’t really explore the system all that much, with 95% of my time in just the basic vi / make / gdb operations. I appreciated the good man pages, as I tried to do everything within the self contained system, without resorting to internet searches. Seeing references to 30+ year old things like Tektronix terminals was amusing.
I was a little surprised that the C++ support wasn’t very good. G++ didn’t support C++11, and LLVM C++ didn’t play nicely with gdb. Gdb crashed on me a lot as well, I suspect due to C++ issues. I know you can get more recent versions through ports, but I stuck with using the base system.
In hindsight, I should have just gone full retro and done everything in ANSI C. I do have plenty of days where, like many older programmers, I think “Maybe C++ isn’t as much of a net positive as we assume...”. There is still much that I like, but it isn’t a hardship for me to build small projects in plain C.
Maybe next time I do this I will try to go full emacs, another major culture that I don’t have much exposure to.
I have a decent overview understanding of most machine learning algorithms, and I have done some linear classifier and decision tree work, but for some reason I have avoided neural networks. On some level, I suspect that Deep Learning being so trendy tweaked a little bit of contrarian in me, and I still have a little bit of a reflexive bias against “throw everything at the NN and let it sort it out!”
In the spirit of my retro theme, I had printed out several of Yann LeCun’s old papers and was considering doing everything completely off line, as if I was actually in a mountain cabin somewhere, but I wound up watching a lot of the Stanford CS231N lectures on YouTube, and found them really valuable. Watching lecture videos is something that I very rarely do — it is normally hard for me to feel the time is justified, but on retreat it was great!
I don’t think I have anything particularly insightful to add about neural networks, but it was a very productive week for me, solidifying “book knowledge” into real experience.
I used a common pattern for me: get first results with hacky code, then write a brand new and clean implementation with the lessons learned, so they both exist and can be cross checked.
I initially got backprop wrong both times, comparison with numerical differentiation was critical! It is interesting that things still train even when various parts are pretty wrong — as long as the sign is right most of the time, progress is often made.
I was pretty happy with my multi-layer neural net code; it wound up in a form that I can just drop it into future efforts. Yes, for anything serious I should use an established library, but there are a lot of times when just having a single .cpp and .h file that you wrote ever line of is convenient.
My conv net code just got to the hacky but working phase, I could have used another day or two to make a clean and flexible implementation.
One thing I found interesting was that when testing on MNIST with my initial NN before adding any convolutions, I was getting significantly better results than the non-convolutional NN reported for comparison in LeCun ‘98 — right around 2% error on the test set with a single 100 node hidden layer, versus 3% for both wider and deeper nets back then. I attribute this to the modern best practices —ReLU, Softmax, and better initialization.
This is one of the most fascinating things about NN work — it is all so simple, and the breakthrough advances are often things that can be expressed with just a few lines of code. It feels like there are some similarities with ray tracing in the graphics world, where you can implement a physically based light transport ray tracer quite quickly, and produce state of the art images if you have the data and enough runtime patience.
I got a much better gut-level understanding of overtraining / generalization / regularization by exploring a bunch of training parameters. On the last night before I had to head home, I froze the architecture and just played with hyperparameters. “Training!” Is definitely worse than “Compiling!” for staying focused.
Now I get to keep my eyes open for a work opportunity to use the new skills!
I am dreading what my email and workspace are going to look like when I get into the office tomorrow.
Stack-register Checking (https://undeadly.org/cgi?action=article;sid=20180310000858)
Recently, Theo de Raadt (deraadt@) described a new type of mitigation he has been working on together with Stefan Kempf (stefan@):
How about we add another new permission! This is not a hardware permission, but a software permission. It is opportunistically enforced by the kernel.
The permission is MAP_STACK. If you want to use memory as a stack, you must mmap it with that flag bit. The kernel does so automatically for the stack region of a process's stack. Two other types of stack occur: thread stacks, and alternate signal stacks. Those are handled
in clever ways.
When a system call happens, we check if the stack-pointer register points to such a page. If it doesn't, the program is killed. We have tightened the ABI. You may no longer point your stack register at non-stack memory. You'll be killed. This checking code is MI, so it works for all platforms.
For more detail, see Theo's original message (https://marc.info/?l=openbsd-tech&m=152035796722258&w=2).
This is now available in snapshots, and people are finding the first problems in the ports tree already. So far, few issues have been uncovered, but as Theo points out, more testing is necessary:
Fairly good results.
A total of 4 problems have been found so far. go, SBCL, and two cases in src/regress which failed the new page-alignment requirement. The SBCL and go ones were found at buildtime, since they use themselves to complete build.
But more page-alignment violations may be found in ports at runtime.
This is something I worry about a bit. So please everyone out there can help: Use snapshots which contain the stack-check diff, update to new packages, and test all possible packages. Really need a lot of testing for this, so please help out.
So, everybody, install the latest snapshot and try all your favorite ports. This is the time to report issues you find, so there is a good chance this additional security feature is present in 6.3 (and works with third party software from packages).
NomadBSD 1.0 has been released (https://freeshell.de/~mk/projects/nomadbsd.html)
NomadBSD is a live system for flash drives, based on FreeBSD® 11.1 (amd64)
Change Log
The setup process has been improved.
Support for optional geli encryption of the home partition has been added
Auto-detection of NVIDIA graphics cards and their corresponding driver has been added. (Thanks to holgerw and lme from BSDForen.de)
An rc script to start the GEOM disk scheduler on the root device has been added.
More software has been added:
accessibility/redshift (starts automatically)
audio/cantata
audio/musicpd
audio/ncmpc
ftp/filezilla
games/bsdtris
mail/neomutt
math/galculator
net-p2p/transmission-qt5
security/fpm2
sysutils/bsdstats
x11/metalock
x11/xbindkeys
Several smaller improvements and bugfixes.
Screenshots
https://freeshell.de/~mk/projects/nomadbsd-ss1.png
https://freeshell.de/~mk/projects/nomadbsd-ss2.png
https://freeshell.de/~mk/projects/nomadbsd-ss3.png
https://freeshell.de/~mk/projects/nomadbsd-ss4.png
https://freeshell.de/~mk/projects/nomadbsd-ss5.png
https://freeshell.de/~mk/projects/nomadbsd-ss6.png
Beastie Bits
KnoxBug - Nagios (http://knoxbug.org/2018-03-27)
vBSDcon videos landing (https://www.youtube.com/playlist?list=PLfJr0tWo35bc9FG_reSki2S5S0G8imqB4)
AsiaBSDCon 2017 videos (https://www.youtube.com/playlist?list=PLnTFqpZk5ebBTyXedudGm6CwedJGsE2Py)
DragonFlyBSD Adds New "Ptr_Restrict" Security Option (https://www.phoronix.com/scan.php?page=news_item&px=DragonFlyBSD-Ptr-Restrict)
A Dexter needs your help (https://twitter.com/michaeldexter/status/975603855407788032)
Mike Larkin at bhyvecon 2018: OpenBSD vmm(4) update (https://undeadly.org/cgi?action=article;sid=20180309064801)
[HEADS UP] - OFED/RDMA stack update (https://lists.freebsd.org/pipermail/freebsd-arch/2018-March/018900.html)
***
Feedback/Questions
Ron - Interview someone using DragonflyBSD (http://dpaste.com/3BM6GSW#wrap)
Brad - Gaming and all (http://dpaste.com/3X4ZZK2#wrap)
Mohammad - Sockets vs TCP (http://dpaste.com/0PJMKRD#wrap)
Paul - All or at least most of Bryan Cantrill's Talks (http://dpaste.com/2WXVR1X#wrap)
***
| |||
| 13 Feb 2025 | 598: UFS1 up-to-date | 01:05:44 | |
Key Considerations for Benchmarking Network Storage Performance, OpenZFS 2.3.0 available, Updates on AsiaBSDcon, GhostBSD Desktop Conference, Recovering from external zroot, Create a new issue in a Github repository with Ansible, Stories I refuse to believe, date limit in UFS1 filesystem extended, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Key Considerations for Benchmarking Network Storage Performance (https://klarasystems.com/articles/considerations-benchmarking-network-storage-performance/)
OpenZFS 2.3.0 available (https://github.com/openzfs/zfs/releases/tag/zfs-2.3.0)
News Roundup
Updates on AsiaBSDCon 2025 - Cancelled - (https://lists.asiabsdcon.org/pipermail/announce/2025-January/000046.html)
GhostBSD Desktop Conference (https://www.phoronix.com/news/BSD-Desktop-Conference-GhostBSD)
Recovering from external zroot (https://adventurist.me/posts/00350)
Create a new issue in a Github repository with Ansible (https://jpmens.net/2025/01/25/create-a-new-issue-in-a-github-repository/)
Stories I refuse to believe (https://flak.tedunangst.com/post/stories-i-refuse-to-believe)
Defer the January 19, 2038 date limit in UFS1 filesystems to February 7, 2106 (https://cgit.freebsd.org/src/commit/?id=1111a44301da39d7b7459c784230e1405e8980f8)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Feedback - Nelson - Ada/GCC (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/598/feedback/Nelson%20Feedback.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 10 Oct 2019 | 319: Lack Rack, Jack | 01:07:50 | |
Causing ZFS corruption for fun, NetBSD Assembly Programming Tutorial, The IKEA Lack Rack for Servers, a new OmniOS Community Edition LTS has been published, List Block Devices on FreeBSD lsblk(8) Style, Project Trident 19.10 available, and more.
Headlines
Causing ZFS corruption for fun and profit (https://datto.engineering/post/causing-zfs-corruption)
Datto backs up data, a lot of it. At the time of writing Datto has over 500 PB of data stored on ZFS. This count includes both backup appliances that are sent to customer sites, as well as cloud storage servers that are used for secondary and tertiary backup of those appliances. At this scale drive swaps are a daily occurrence, and data corruption is inevitable. How we handle this corruption when it happens determines whether we truly lose data, or successfully restore from secondary backup. In this post we'll be showing you how at Datto we intentionally cause corruption in our testing environments, to ensure we're building software that can properly handle these scenarios.
Causing Corruption
Since this is a mirror setup, a naive solution to cause corruption would be to randomly dd the same sectors of both /dev/sdb and /dev/sdc. This works, but is equally likely to just overwrite random unused space, or take down the zpool entirely. What we really want is to corrupt a specific snapshot, or even a specific file in that snapshot, to simulate a more realistic minor corruption event. Luckily we have a tool called zdb that lets us view some low level information about datasets.
Conclusion
At the 500 PB scale, it's not a matter of if data corruption will happen but when. Intentionally causing corruption is one of the strategies we use to ensure we're building software that can handle these rare (but inevitable) events.
To others out there using ZFS: I'm curious to hear how you've solved this problem. We did quite a bit of experimentation with zinject before going with this more brute force method. So I'd be especially interested if you've had luck simply simulating corruption with zinject.
NetBSD Assembly Programming Tutorial (https://polprog.net/blog/netbsdasmprog/)
A sparc64 version is also being prepared and will be added when done
This post describes how to write a simple hello world program in pure assembly on NetBSD/amd64. We will not use (nor link against) libc, nor use gcc to compile it. I will be using GNU as (gas), and therefore the AT&T syntax instead of Intel.
Why assembly?
Why not? Because it's fun to program in assembly directly. Contrary to a popular belief assembly programs aren't always faster than what optimizing compilers produce. Nevertheless it's good to be able to read assembly, especially when debugging C programs
Due to the nature of the guide, visit the site for the complete breakdown
News Roundup
The IKEA Lack Rack for Servers (https://wiki.eth0.nl/index.php/LackRack)
The LackRack
First occurrence on eth0:2010 Winterlan, the LackRack is the ultimate, low-cost, high shininess solution for your modular datacenter-in-the-living-room. Featuring the LACK (side table) from Ikea, the LackRack is an easy-to-implement, exact-fit datacenter building block. It's a little known fact that we have seen Google engineers tinker with Lack tables since way back in 2009.
The LackRack will certainly make its appearance again this summer at eth0:2010 Summer.
Summary
When temporarily not in use, multiple LackRacks can be stacked in a space-efficient way without disassembly, unlike competing 19" server racks.
The LackRack was first seen on eth0:2010 Winterlan in the no-shoe Lounge area. Its low-cost and perfect fit are great for mounting up to 8 U of 19" hardware, such as switches (see below), or perhaps other 19" gear. It's very easy to assemble, and thanks to the design, they are stable enough to hold (for example) 19" switches and you can put your bottle of Club-Mate on top! Multi-shiny LackRack can also be painted to your specific preferences and the airflow is unprecedented!
Howto
You can find a howto on buying a LackRack on this page. This includes the proof that a 19" switch can indeed be placed in the LackRack in its natural habitat!
OmniOS Community Edition r151030 LTS - Published at May 6, 2019 (https://omniosce.org/article/release-030)
The OmniOS Community Edition Association is proud to announce the general availability of OmniOS - r151030.
OmniOS is published according to a 6-month release cycle, r151030 LTS takes over from r151028, published in November 2018; and since it is a LTS release it also takes over from r151022. The r151030 LTS release will be supported for 3 Years. It is the first LTS release published by the OmniOS CE Association since taking over the reins from OmniTI in 2017. The next LTS release is scheduled for May 2021. The old stable r151026 release is now end-of-life. See the release schedule for further details.
This is only a small selection of the new features, and bug fixes in the new release; review the release notes for full details.
If you upgrade from r22 and want to see all new features added since then, make sure to also read the release notes for r24, r26 and r28.
For full relase notes including upgrade instructions;
release notes (https://omniosce.org/releasenotes.html)
upgrade instructions (https://omniosce.org/upgrade.html)
List Block Devices on FreeBSD lsblk(8) Style (https://vermaden.wordpress.com/2019/09/27/list-block-devices-on-freebsd-lsblk8-style/)
When I have to work on Linux systems I usually miss many nice FreeBSD tools such as these for example to name the few: sockstat, gstat, top -b -o res, top -m io -o total, usbconfig, rcorder, beadm/bectl, idprio/rtprio,… but sometimes – which rarely happens – Linux has some very useful tool that is not available on FreeBSD. An example of such tool is lsblk(8) that does one thing and does it quite well – lists block devices and their contents. It has some problems like listing a disk that is entirely used under ZFS pool on which lsblk(8) displays two partitions instead of information about ZFS just being there – but we all know how much in some circles the CDDL licensed ZFS is unloved in that GPL world.
Example lsblk(8) output from Linux system:
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sr0 11:0 1 1024M 0 rom
sda 8:0 0 931.5G 0 disk
|-sda1 8:1 0 500M 0 part /boot
`-sda2 8:2 0 931G 0 part
|-vg_local-lv_root (dm-0) 253:0 0 50G 0 lvm /
|-vg_local-lv_swap (dm-1) 253:1 0 17.7G 0 lvm [SWAP]
`-vg_local-lv_home (dm-2) 253:2 0 1.8T 0 lvm /home
sdc 8:32 0 232.9G 0 disk
`-sdc1 8:33 0 232.9G 0 part
`-md1 9:1 0 232.9G 0 raid10 /data
sdd 8:48 0 232.9G 0 disk
`-sdd1 8:49 0 232.9G 0 part
`-md1 9:1 0 232.9G 0 raid10 /data
What FreeBSD offers in this department? The camcontrol(8) and geom(8) commands are available. You can also use gpart(8) command to list partitions. Below you will find output of these commands from my single disk laptop. Please note that because of WordPress limitations I need to change all > < characters to ] [ ones in the commands outputs.
See the article for the rest of the guide
Project Trident 19.10 Now Available (https://project-trident.org/post/2019-10-05_19.10_available/)
This is a general package update to the CURRENT release repository based upon TrueOS 19.10
PACKAGE CHANGES FROM 19.08
New Packages: 601
Deleted Packages: 165
Updated Packages: 3341
Beastie Bits
NetBSD building tools (https://imgur.com/gallery/0sG4b1K)
Sponsorships open for SNMP Mastery (https://mwl.io/archives/4569)
pkgsrc-2019Q3 release announcement (2019-10-03) (http://mail-index.netbsd.org/pkgsrc-users/2019/10/03/msg029485.html)
pfetch - A simple system information tool written in POSIX sh (https://github.com/dylanaraps/pfetch)
Taking NetBSD kernel bug roast to the next level: Kernel Fuzzers (quick A.D. 2019 overview) (https://netbsd.org/~kamil/eurobsdcon2019_fuzzing/presentation.html#slide1)
Cracking Ken Thomson’s password (https://leahneukirchen.org/blog/archive/2019/10/ken-thompson-s-unix-password.html)
Feedback/Questions
Evilham - Couple Questions (http://dpaste.com/2JC85WV)
Rob - APU2 alternatives and GPT partition types (http://dpaste.com/0SDX9ZX)
Tom - FreeBSD journal article by A. Fengler (http://dpaste.com/2B43MY1#wrap)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 05 Nov 2014 | 62: Gift from the Sun | 00:34:08 | |
We're away at MeetBSD this week, but we've still got a great show for you. We'll be joined by Pawel Dawidek, who's done quite a lot of things in FreeBSD over the years, including the initial ZFS port. We'll get to hear how that came about, what he's up to now and a whole lot more. We'll be back next week with a normal episode of BSD Now - the place to B.. SD.
This episode was brought to you by
Interview - Pawel Jakub Dawidek - pjd@freebsd.org (mailto:pjd@freebsd.org)
Porting ZFS, GEOM, GELI, Capsicum, various topics
| |||
| 11 Sep 2013 | 2: Engineering and Powder Kegs | 01:40:54 | |
After a wildly successful debut episode, BSD Now is BACK to talk with Glen Barber from the FreeBSD Release team, show you how to build your own binary package repository and discuss the latest BSD news!
Headlines
64bit time in OpenBSD (http://undeadly.org/cgi?action=article&sid=20130813072244)
Many operating systems face an upcoming challenge, similar to (but more complicated than) Y2K: Y2038. All of the BSDs and most other operating systems track time by counting the seconds since Jan 1st, 1970. In 2038 this value will reach the maximum value of a signed 32 bit integer.
Simply changing to a 64 bit counter may not be the best solution, because there may still be 32 bit systems in use for embedded applications
Theo will be giving the keynote at EuroBSDCon on the subject, explaining how OpenBSD has implemented the solution
ABI incompatibility. Updating to this kernel requires extra work or you won't be able to login: install a snapshot instead. Upgrading by source is for the insane only. (http://www.openbsd.org/faq/current.html#20130813)
AESNI pipelining gets a speed boost (http://freshbsd.org/commit/freebsd/r255187)
AES-NI is a new processor instruction available on modern Intel and AMD chips that provides hardware acceleration for AES encryption and decryption. This feature is especially useful for encrypted disks, because it removes most of the performance penalty traditionally associated with encryption
The new commit has the instructions pipelined, so there is no latency between the instructions
Uses SSE2 instructions for calculating XTS tweak factor for further increased performance
GELI based disk encryption performance increased by 3x on capable CPUs
Should affect PEFS and other AES backed encryption schemes as well
Full disk encryption should be more or less transparent now
***
OpenBSD 5.4 Preorders (http://openbsd.org/orders.html)
Every 6 months there is a new OpenBSD version
They include a fun song and nicely-packaged CD set
The proceeds from sale of these products is the primary funding of the OpenBSD project
The official ISOs will be uploaded on November 1st
***
GCC no longer built by default on FreeBSD -CURRENT (https://svnweb.freebsd.org/base?view=revision&revision=255321)
On platforms where clang is the default compiler, don't build gcc or libstdc++
GCC is still enabled on PC98, because the PC98 bootloader requires GCC to build
While the base FreeBSD system has been built by clang for a long time, this change also covers the ports tree
***
Patch to update Xorg and MESA on FreeBSD (http://lists.freebsd.org/pipermail/freebsd-x11/2013-September/013599.html)
Updates xorg drivers
Expected to be committed in about 2 weeks
Adds option to use devd instead of HAL for X configuration
Updates the MESA stack (9.1.6), libGL, DRI, etc
Enables KMS for AMD/ATI cards
Mesa 9.2 is available with xorg-dev, OpenBSD has also recently upgraded to Mesa 9.2 for their stable version of Xorg
***
Interview - Glen Barber - gjb@freebsd.org (mailto:gjb@freebsd.org) / @evilgjb (https://twitter.com/evilgjb)
FreeBSD Release Engineering
Tutorial
Making your own binary repository (http://www.bsdnow.tv/tutorials/poudriere)
The Place to B...SD
iXsystems hosts FreeBSD Anniversary party (http://www.ixsystems.com/resources/ix/news/ixsystems-to-once-again-host-freebsd-anniversary-celebration.html)
Celebrating FreeBSD’s 20th anniversary
Saturday, November 2nd at the DNA Lounge in San Francisco
Notable FreeBSD figures will contribute words of wisdom on the past, present, and future of FreeBSD
***
News Roundup
NetBSD gets basic support for the cubieboard 1 & 2 (http://mail-index.netbsd.org/source-changes/2013/09/04/msg047155.html)
Very preliminary support for cubieboard 1 & 2 based on the Allwinner A10 & A20 SoCs
Many drivers are "stubs with autoconf glue"
Contributed by Matt Thomas
***
Rayservers ditches Linux for BSD (http://rayservers.com/blog/the-freebsd-daemon-is-off-to-do-battle-in-the-name-of-christ)
Used them all, Windows, Mac, OpenBSD, Linux
Needed PF, ZFS, disk encryption, lots of networking features, better security
In Linux, "The new cgroups based memory management ran out of memory - on a 256 GB RAM system whilst it was not using more than 40."
BSD now protects the privacy of their email users
***
HPN for OpenSSH 6.2 (https://lists.mindrot.org/pipermail/openssh-unix-dev/2013-August/031614.html)
High Performance Networking (http://www.psc.edu/index.php/hpn-ssh) is an SSH patchset to improve transfer speeds by removing the fixed window size and take better advantage of TCP
Maintained as a patchset separate from OpenSSH
First integrated into FreeBSD base as of 9.0
Updated to support 6.2 (available in the ports tree as security/openssh-portable)
The HPN patch set also includes threaded AES-CTR support to increase performance and take advantage of multiple CPU cores for encryption. In this latest patch, threaded AES-CTR now works in all situations (it failed in some specific situations previously). Expected performance increase is ~50%
NONE cipher is now separate from the main patch set. The NONE cipher allows tools like scp and sftp to switch off the encryption for file transfers (when specifically told to do so) to keep encryption from bottlenecking performance and wasting CPU time
***
Call for testing: OpenSSH-6.3 (https://lists.mindrot.org/pipermail/openssh-unix-dev/2013-July/031550.html)
Mostly a bugfix release
SFTP now supports resuming partially-downloaded or uploaded transfers
More logging features
Six weeks after the initial email, still no release. des@ is not pleased.
***
pkgsrc gets signing support (http://mail-index.netbsd.org/pkgsrc-users/2013/08/30/msg018511.html)
pkgsrc is used on NetBSD, DragonflyBSD and other OSes
Comes from an EdgeBSD developer
Uses GPG for signing package files
Currently just a patch on github and in its infancy
Provides a short howto
***
FreeBSD vs. Linux: 10 points of superiority (https://forums.freebsd.org/showthread.php?t=41750)
New FreeBSD user, ex-Linux user writes about his experience
Mentions consistency, documentation, security, filesystems, updates, jails, community
Really long post, definitely worth a read
***
Feedback/Questions
hoopla writes in (http://slexy.org/view/s21SpCcisW)
Juergen writes in (http://slexy.org/view/s20vHY9qAK)
Sam writes in (http://slexy.org/view/s23uf4vzfQ)
Frank writes in (http://slexy.org/view/s2Y0qiXJan)
***
| |||
| 06 Jul 2023 | 514: Infecting Public Keys | 00:48:30 | |
OpenZFS, Your Data and the Challenge of Ransomware, I Didn’t Learn Unix By Reading All The Manpages, I try to answer "how to become a systems engineer", Writing shell scripts in Nushell, Sudo and signal propagation, infecting SSH Public Keys with backdoors, OpenBSD Thinkpad, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
OpenZFS, Your Data and the Challenge of Ransomware (https://klarasystems.com/articles/openzfs-openzfs-your-data-and-the-challenge-of-ransomware/)
I Didn’t Learn Unix By Reading All The Manpages (https://www.owlfolio.org/research/i-didnt-learn-unix-by-reading-all-the-manpages/)
News Roundup
Feedback: I try to answer "how to become a systems engineer" (https://rachelbythebay.com/w/2023/05/30/eng/)
Writing shell scripts in Nushell (https://jpospisil.com/2023/05/25/writing-shell-scripts-in-nushell)
Sudo and signal propagation (https://dxuuu.xyz/sudo.html)
Infecting SSH Public Keys with backdoors (https://blog.thc.org/infecting-ssh-public-keys-with-backdoors)
OpenBSD Thinkpad (https://douglasrumbaugh.com/post/openbsd-thinkpad-good/)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 18 Jul 2019 | 307: Twitching with OpenBSD | 00:50:59 | |
FreeBSD 11.3 has been released, OpenBSD workstation, write your own fuzzer for the NetBSD kernel, Exploiting FreeBSD-SA-19:02.fd, streaming to twitch using OpenBSD, 3 different ways of dumping hex contents of a file, and more.
Headlines
FreeBSD 11.3-RELEASE Announcement (https://www.freebsd.org/releases/11.3R/announce.html)
The FreeBSD Release Engineering Team is pleased to announce the availability of FreeBSD 11.3-RELEASE. This is the fourth release of the stable/11 branch.
Some of the highlights:
The clang, llvm, lld, lldb, and compiler-rt utilities as well as libc++ have been updated to upstream version 8.0.0.
The ELF Tool Chain has been updated to version r3614.
OpenSSL has been updated to version 1.0.2s.
The ZFS filesystem has been updated to implement parallel mounting.
The loader(8) has been updated to extend geli(8) support to all architectures.
The pkg(8) utility has been updated to version 1.10.5.
The KDE desktop environment has been updated to version 5.15.3.
The GNOME desktop environment has been updated to version 3.28.
The kernel will now log the jail(8) ID when logging a process exit.
Several feature additions and updates to userland applications.
Several network driver firmware updates.
Warnings for features deprecated in future releases will now be printed on all FreeBSD versions.
Warnings have been added for IPSec algorithms deprecated in RFC 8221.
Deprecation warnings have been added for weaker algorithms when creating geli(8) providers.
And more...
OpenBSD Is Now My Workstation (https://sogubsys.com/openbsd-is-now-my-workstation-operating-system/)
Why OpenBSD? Simply because it is the best tool for the job for me for my new-to-me Lenovo Thinkpad T420. Additionally, I do care about security and non-bloat in my personal operating systems (business needs can have different priorities, to be clear).
I will try to detail what my reasons are for going with OpenBSD (instead of GNU/Linux, NetBSD, or FreeBSD of which I’m comfortable using without issue), challenges and frustrations I’ve encountered, and what my opinions are along the way.
Disclaimer: in this post, I’m speaking about what is my opinion, and I’m not trying to convince you to use OpenBSD or anything else. I don’t truly care, but wanted to share in case it could be useful to you. I do hope you give OpenBSD a shot as your workstation, especially if it has been a while.
A Bit About Me and OpenBSD
I’m not new to OpenBSD, to be clear. I’ve been using it off and on for over 20 years. The biggest time in my life was the early 2000s (I was even the Python port maintainer for a bit), where I not only used it for my workstation, but also for production servers and network devices.
I just haven’t used it as a workstation (outside of a virtual machine) in over 10 years, but have used it for servers. Workstation needs, especially for a primary workstation, are greatly different and the small things end up mattering most.
News Roundup
Write your own fuzzer for NetBSD kernel! [Part 1] (https://blog.netbsd.org/tnf/entry/write_your_own_fuzzer_for)
How Fuzzing works? The dummy Fuzzer.
The easy way to describe fuzzing is to compare it to the process of unit testing a program, but with different input. This input can be random, or it can be generated in some way that makes it unexpected form standard execution perspective.
The simplest 'fuzzer' can be written in few lines of bash, by getting N bytes from /dev/rand, and putting them to the program as a parameter.
Coverage and Fuzzing
What can be done to make fuzzing more effective? If we think about fuzzing as a process, where we place data into the input of the program (which is a black box), and we can only interact via input, not much more can be done.
However, programs usually process different inputs at different speeds, which can give us some insight into the program's behavior. During fuzzing, we are trying to crash the program, thus we need additional probes to observe the program's behaviour.
Additional knowledge about program state can be exploited as a feedback loop for generating new input vectors. Knowledge about the program itself and the structure of input data can also be considered. As an example, if the input data is in the form of HTML, changing characters inside the body will probably cause less problems for the parser than experimenting with headers and HTML tags.
For open source programs, we can read the source code to know what input takes which execution path. Nonetheless, this might be very time consuming, and it would be much more helpful if this can be automated. As it turns out, this process can be improved by tracing coverage of the execution
vBSDcon - CFP - Call for Papers ends July 19th (https://vbsdcon.com/)
You can submit your proposal at https://easychair.org/conferences/?conf=vbsdcon2019
The talks will have a very strong technical content bias. Proposals of a business development or marketing nature are not appropriate for this venue.
If you are doing something interesting with a BSD operating system, please submit a proposal. Whether you are developing a very complex system using BSD as the foundation, or helping others and have a story to tell about how BSD played a role, we want to hear about your experience. People using BSD as a platform for research are also encouraged to submit a proposal.
Possible topics include: How we manage a giant installation with respect to handling spam, snd/or sysadmin, and/or networking, Cool new stuff in BSD, Tell us about your project which runs on BSD.
Both users and developers are encouraged to share their experiences.
Exploiting FreeBSD-SA-19:02.fd (https://secfault-security.com/blog/FreeBSD-SA-1902.fd.html)
In February 2019 the FreeBSD project issued an advisory about a possible vulnerability in the handling of file descriptors. UNIX-like systems such as FreeBSD allow to send file descriptors to other processes via UNIX-domain sockets. This can for example be used to pass file access privileges to the receiving process.
Inside the kernel, file descriptors are used to indirectly reference a C struct which stores the relevant information about the file object. This could for instance include a reference to a vnode which describes the file for the file system, the file type, or the access privileges.
What really happens if a UNIX-domain socket is used to send a file descriptor to another process is that for the receiving process, inside the kernel a reference to this struct is created. As the new file descriptor is a reference to the same file object, all information is inherited. For instance, this can allow to give another process write access to a file on the drive even if the process owner is normally not able to open the file writable.
The advisory describes that FreeBSD 12.0 introduced a bug in this mechanism. As the file descriptor information is sent via a socket, the sender and the receiver have to allocate buffers for the procedure. If the receiving buffer is not large enough, the FreeBSD kernel attempts to close the received file descriptors to prevent a leak of these to the sender. However, while the responsible function closes the file descriptor, it fails to release the reference from the file descriptor to the file object. This could cause the reference counter to wrap.
The advisory further states that the impact of this bug is possibly a local privilege escalation to gain root privileges or a jail escape. However, no proof-of-concept was provided by the advisory authors.
In the next section, the bug itself is analyzed to make a statement about the bug class and a guess about a possible exploitation primitive.
After that, the bug trigger is addressed.
It follows a discussion of three imaginable exploitation strategies - including a discussion of why two of these approaches failed.
In the section before last, the working exploit primitive is discussed. It introduces a (at least to the author’s knowledge) new exploitation technique for these kind of vulnerabilities in FreeBSD. The stabilization of the exploit is addressed, too.
The last section wraps everything up in a conclusion and points out further steps and challenges.
The privilege escalation is now a piece of cake thanks to a technique used by kingcope, who published a FreeBSD root exploit in 2005, which writes to the file /etc/libmap.conf. This configuration file can be used to hook the loading of dynamic libraries if a program is started. The exploit therefore creates a dynamic library, which copies /bin/sh to another file and sets the suid-bit for the copy. The hooked library is libutil, which is for instance called by su. Therefore, a call to su by the user will afterwards result in a suid copy of /bin/sh.
Streaming to Twitch using OpenBSD (https://dataswamp.org/~solene/2019-07-06-twitch.html)
Introduction
If you ever wanted to make a twitch stream from your OpenBSD system, this is now possible, thanks to OpenBSD developer thfr@ who made a wrapper named fauxstream using ffmpeg with relevant parameters.
The setup is quite easy, it only requires a few steps and searching on Twitch website two informations, hopefully, to ease the process, I found the links for you.
You will need to make an account on twitch, get your api key (a long string of characters) which should stay secret because it allow anyone having it to stream on your account.
These same techniques should work for Twitch, YouTube Live, Periscope, Facebook, etc, including the live streaming service ScaleEngine provides free to BSD user groups.
There is also an open source application called ‘OBS’ or Open Broadcaster Studio. It is in FreeBSD ports and should work on all of the other BSDs as well. It has a GUI and supports compositing and green screening. We use it heavily at ScaleEngine and it is also used at JupiterBroadcasting in place of WireCast, a $1000-per-copy commercial application.
Beastie Bits
Portland BSD Pizza Night - 2019-07-25 19:00 - Rudy's Gourmet Pizza (http://calagator.org/events/1250475868)
KnoxBUG - Michael W. Lucas : Twenty Years in Jail (http://knoxbug.org/2019-07-29)
Ohio Linuxfest - CFP - Closes August 17th (https://ohiolinux.org/call-for-presentations/)
My college (NYU Tandon) is moving their CS department and I saw this on a shelf being moved (https://old.reddit.com/r/freebsd/comments/cdx8fp/my_college_nyu_tandon_is_moving_their_cs/)
3 different ways of dumping hex contents of a file (https://moopost.blogspot.com/2019/07/3-different-ways-of-dumping-hex.html)
Feedback/Questions
Sebastian - ZFS setup toward ESXi (http://dpaste.com/0DRKFH6#wrap)
Christopher - Questions (http://dpaste.com/2YNN1SH)
Ser - Bhyve and Microsoft SQL (http://dpaste.com/1F5TMT0#wrap)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 27 Oct 2022 | 478: Debunking sudo myths | 00:46:13 | |
Open Source in Enterprise Environments, Your Comprehensive Guide to rc(8): FreeBSD Services and Automation, How Rob Pike got hired by Dennis Richie, what FreeBSD machines rubenerd uses, new debugbreak command, 7 sudo myths debunked
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Open Source in Enterprise Environments - Where Are We Now and What Is Our Way Forward? (https://bsdly.blogspot.com/2022/09/open-source-in-enterprise-environments.html)
Your Comprehensive Guide to rc(8): FreeBSD Services and Automation (https://klarasystems.com/articles/rc8-freebsd-services-and-automation/)
News Roundup
How Rob Pike got hired by Dennis Richie (https://minnie.tuhs.org/pipermail/tuhs/2022-September/026506.html)
Cartron asks what FreeBSD machines I use (https://rubenerd.com/cartron-asks-what-freebsd-machines-i-use/)
My new debugbreak command (https://nullprogram.com/blog/2022/07/31/)
7 sudo myths debunked (https://opensource.com/article/22/8/debunk-sudo-myths)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Andy - sharing and acls (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/478/feedback/Andy%20-%20sharing%20and%20acls.md)
Reptilicus Rex - boot environments (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/478/feedback/Reptilicus%20Rex%20-%20boot%20environments.md)
i3luefire - byhve issue (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/478/feedback/i3luefire%20-%20byhve%20issue.md)
***
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 03 May 2017 | 192: SSHv1 Be Gone | 02:04:13 | |
This week we have a FreeBSD Foundation development update, tell you about sprinkling in the TrueOS project, Dynamic WDS & a whole lot more!
This episode was brought to you by
Headlines
OpenSSH Removes SSHv1 Support (http://undeadly.org/cgi?action=article&sid=20170501005206)
In a series of commits starting here (http://marc.info/?l=openbsd-cvs&m=149359384905651&w=2) and ending with this one (http://marc.info/?l=openbsd-cvs&m=149359530105864&w=2), Damien Miller completed the removal of all support for the now-historic SSHv1 protocol from OpenSSH (https://www.openssh.com/).
The final commit message, for the commit that removes the SSHv1 related regression tests, reads:
Eliminate explicit specification of protocol in tests and loops over protocol. We only support SSHv2 now.
Dropping support for SSHv1 and associated ciphers that were either suspected to or known to be broken has been planned for several releases, and has been eagerly anticipated by many in the OpenBSD camp.
In practical terms this means that starting with OpenBSD-current and snapshots as they will be very soon (and further down the road OpenBSD 6.2 with OpenSSH 7.6), the arcane options you used with ssh (http://man.openbsd.org/ssh) to connect to some end-of-life gear in a derelict data centre you don't want to visit anymore will no longer work and you will be forced do the reasonable thing. Upgrade.
FreeBSD Foundation April 2017 Development Projects Update (https://www.freebsdfoundation.org/blog/april-2017-development-projects-update/)
FreeBSD runs on many embedded boards that provide a USB target or USB On-the-Go (OTG) interface. This allows the embedded target to act as a USB device, and present one or more interfaces (USB device classes) to a USB host. That host could be running FreeBSD, Linux, Mac OS, Windows, Android, or another operating system. USB device classes include audio input or output (e.g. headphones), mass storage (USB flash drives), human interface device (keyboards, mice), communications (Ethernet adapters), and many others.
The Foundation awarded a project grant to Edward Tomasz Napierała to develop a USB mass storage target driver, using the FreeBSD CAM Target Layer (CTL) as a backend. This project allows FreeBSD running on an embedded platform, such as a BeagleBone Black or Raspberry Pi Zero, to emulate a USB mass storage target, commonly known as a USB flash stick. The backing storage for the emulated mass storage target is on the embedded board’s own storage media. It can be configured at runtime using the standard CTL configuration mechanism – the ctladm(8) utility, or the ctl.conf(5) file.
The FreeBSD target can now present a mass storage interface, a serial interface (for a console on the embedded system), and an Ethernet interface for network access. A typical usage scenario for the mass storage interface is to provide users with documentation and drivers that can be accessed from their host system. This makes it easier for new users to interact with the embedded FreeBSD board, especially in cases where the host operating system may require drivers to access all of the functionality, as with Windows and OS X.
They provide instructions on how to configure a BeagleBone Black to act as a flash memory stick attached to a host computer.
+Check out the article, test, and report back your experiences with the new USB OTG interface.
***
Spring cleaning: Hardware Update and Preview of upcoming TrueOS changes (https://www.trueos.org/blog/spring-cleaning-hardware-update-preview-upcoming-trueos-changes/)
The much-abused TrueOS build server is experiencing some technical difficulties, slowing down building new packages and releasing updates. After some investigation, one problem seemed to be a bug with the Poudriere port building software. After updating builders to the new version, some of the instability is resolved. Thankfully, we won’t have to rely on this server so much, because…
We’re getting new hardware!
A TrueOS/Lumina contributor is donating a new(ish) server to the project. Special thanks to TrueOS contributor/developer q5sys for the awesome new hardware!
Preview: UNSTABLE and Upcoming TrueOS STABLE update
A fresh UNSTABLE release is dropping today, with a few key changes:
Nvidia/graphics driver detection fixes.
Boot environment listing fix (FreeBSD boot-loader only)
Virtual box issues fixed on most systems. There appears to be a regression in VirtualBox 5.1 with some hardware.
New icon themes for Lumina (Preferences -> Appearance -> Theme).
Removal of legacy pc-diskmanager. It was broken and unmaintained, so it is time to remove it.
Installer/.iso Changes (Available with new STABLE Update):
The text installer has been removed. It was broken and unmaintained, so it is time to remove it.
There is now a single TrueOS install image. You can still choose to install as either a server or desktop, but both options live in a single install image now. This image is still available as either an .iso or .img file.
The size of the .iso and .img files is reduced about 500 Mb to around 2Gb total. We’ve removed Firefox and Thunderbird from the default desktop installation. These have been replaced with Qupzilla and Trojita. Note you can replace Qupzilla and Trojita with Firefox and Thunderbird via the SysAdm Appcafe after completing the TrueOS install.
Grub is no longer an installation option. Instead, the FreeBSD boot-loader is always used for the TrueOS partition. rEFInd is used as the master boot-loader for multi-booting; EFI partitioning is required.
Qpdfview is now preinstalled for pdf viewing.
Included a slideshow during the installation with tips and screenshots.
Interview - Patrick M. Hausen - hausen@punkt.de (mailto:hausen@punkt.de)
Founder of Punkt.de
HAST - Highly Available Storage (https://wiki.freebsd.org/HAST)
News Roundup
(finally) investigating how to get dynamic WDS (DWDS) working in FreeBSD! (http://adrianchadd.blogspot.com/2017/04/finally-investigating-how-to-get.html)
Adrian Chadd writes in his blog:
I sat down recently to figure out how to get dynamic WDS working on FreeBSD-HEAD. It's been in FreeBSD since forever, and it in theory should actually have just worked, but it's extremely not documented in any useful way. It's basically the same technology in earlier Apple Airports (before it grew into what the wireless tech world calls "Proxy-STA") and is what the "extender" technology on Qualcomm Atheros chipsets implement.
A common question I get from people is "why can't I bridge multiple virtual machines on my laptop and have them show up over wifi? It works on ethernet!" And my response is "when I make dynamic WDS work, you can just make this work on FreeBSD devices - but for now, use NAT." That always makes people sad.
+ Goes on to explain that normal station/access point setups have up to three addresses and depending on the packet type, these can vary. There are a couple of variations in the addresses, which is more than the number of address fields in a normal 802.11 frame.
The big note here is that there's not enough MAC addresses to say "please send this frame to a station MAC address, but then have them forward it to another MAC address attached behind it in a bridge." That would require 4 mac addresses in the 802.11 header, which we don't get.
.. except we do. There's a separate address format where from-DS and to-DS bits in the header set to 1, which means "this frame is coming from distribution system to a distribution system", and it has four mac addresses. The RA is then the AP you're sending to, and then a fourth field indicates the eventual destination address that may be an ethernet device connected behind said STA.
If you don't configure up WDS, then when you send frames from a station from a MAC address that isn't actually your 802.11 interface MAC address, the system would be confused. The STA wouldn't be able to transmit it easily, and the AP wouldn't know how to get back to your bridged ethernet addresses.
The original WDS was a statically configured thing. [...] So for static configurations, this works great. You'd associate your extender AP as a station of the central AP, it'd use wpa_supplicant to setup encryption, then anything between that central AP and that extender AP (as a station) would be encrypted as normal station traffic (but, 4-address frame format.)
But that's not very convenient. You have to statically configure everything, including telling your central AP about all of your satellite extender APs. If you want to replace your central AP, you have to reprogram all of your extenders to use the new MAC addresses.
So, Sam Leffler came up with "dynamic WDS" - where you don't have to explicitly state the list of central/satellite APs. Instead, you configure a central AP as "dynamic WDS", and when a 4-address frame shows up from an associated station, it "promotes" it to a WDS peer for you. On the satellite AP, it will just find an AP to communicate to, and then assume it'll do WDS and start using 4-address frames. It's still a bit clunky (there's no beacon, probe request, etc IEs that say "I do dynamic WDS!" so you'd better make ALL your central APs a different SSID!) but it certainly is better than what we had.
Firstly, there are scripts in src/tools/tools/net80211/ - setup.wdsmain and setup.wdsrelay. These scripts are .. well, the almost complete documentation on a dynamic WDS setup. The manpage doesn't go into anywhere near enough information.
So I dug into it. It turns out that dynamic WDS uses a helper daemon - 'wlanwds' - which listens for dynamic WDS configuration changes and will do things for you. This is what runs on the central AP side. Then it started making sense!
So far, so good. I followed that script, modified it a bit to use encryption, and .. well, it half worked. Association worked fine, but no traffic was passing.
A little more digging showed the actual problem - the dynamic WDS example scripts are for an open/unencrypted network. If you are using an encrypted network, the central AP side needs to enable privacy on the virtual interfaces so traffic gets encrypted with the parent interface encryption keys.
Now, I've only done enough testing to show that indeed it is working. I haven't done anything like pass lots of traffic via iperf, or have a mix of DWDS and normal STA peers, nor actually run it for longer than 5 minutes. I'm sure there will be issues to fix. However - I do need it at home, as I can't see the home AP from the upstairs room (and now you see why I care about DWDS!) and so when I start using it daily I'll fix whatever hilarity ensues.
Why don't schools teach debugging? (https://danluu.com/teach-debugging/)
A friend of mine and I couldn’t understand why some people were having so much trouble; the material seemed like common sense. The Feynman Method was the only tool we needed.
Write down the problem
Think real hard
Write down the solution
The Feynman Method failed us on the last project: the design of a divider, a real-world-scale project an order of magnitude more complex than anything we’d been asked to tackle before.
I understand now why half the class struggled with the earlier assignments. Without an explanation of how to systematically approach problems, anyone who didn’t intuitively grasp the correct solution was in for a semester of frustration. People who were, like me, above average but not great, skated through most of the class and either got lucky or wasted a huge chunk of time on the final project. I’ve even seen people talented enough to breeze through the entire degree without ever running into a problem too big to intuitively understand; those people have a very bad time when they run into a 10 million line codebase in the real world. The more talented the engineer, the more likely they are to hit a debugging wall outside of school.
It’s one of the most fundamental skills in engineering: start at the symptom of a problem and trace backwards to find the source. It takes, at most, half an hour to teach the absolute basics – and even that little bit would be enough to save a significant fraction of those who wash out and switch to non-STEM majors.
Why do we leave material out of classes and then fail students who can’t figure out that material for themselves? Why do we make the first couple years of an engineering major some kind of hazing ritual, instead of simply teaching people what they need to know to be good engineers? For all the high-level talk about how we need to plug the leaks in our STEM education pipeline, not only are we not plugging the holes, we’re proud of how fast the pipeline is leaking.
FreeBSD: pNFS server for testing (https://lists.freebsd.org/pipermail/freebsd-fs/2017-April/024702.html)
Rick Macklem has issued a call for testing his new pNFS server:
I now have a pNFS server that I think is ready for testing/evaluation. It is basically a patched FreeBSD-current kernel plus nfsd daemon.
If you are interested, some very basic notes on how it works and how to set it up are at:
http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt (http://people.freebsd.org/~rmacklem/pnfs-planb-setup.txt)
A Plan B pNFS service consists of a single MetaData Server (MDS) and K Data Servers (DS), all of which would be recent FreeBSD systems. Clients will mount the MDS as they would a single NFS server. When files are created, the MDS creates a file tree identical to what a single NFS server creates, except that all the regular (VREG) files will be empty. As such, if you look at the exported tree on the MDS directly on the MDS server (not via an NFS mount), the files will all be of size == 0. Each of these files will also have two extended attributes in the system attribute name space:
pnfsd.dsfile - This extended attrbute stores the information that the MDS needs to find the data storage file on a DS for this file.
pnfsd.dsattr - This extended attribute stores the Size, ModifyTime and Change attributes for the file.
For each regular (VREG) file, the MDS creates a data storage file on one of the K DSs, in one of the dsNN directories. The name of this file is the file handle of the file on the MDS in hexadecimal. The DSs use 20 subdirectories named "ds0" to "ds19" so that no one directory gets too large.
At this time, the MDS generates File Layout layouts to NFSv4.1 clients that know how to do pNFS.
For NFS clients that do not support NFSv4.1 pNFS, there will be a performance hit, since the IO RPCs will be proxied by the MDS for the DS server the data storage file resides on.
The current setup does not allow for redundant servers. If the MDS or any of the K DS servers fail, the entire pNFS service will be non-functional.
Looking at creating mirrored DS servers is planned, but it may be a year or more before that is implemented. I am planning on using the Flex File Layout for this, since it supports client side mirroring, where the client will write to all mirrors concurrently.
Beastie Bits
Openbsd changes of note 620 (http://www.tedunangst.com/flak/post/openbsd-changes-of-note-620)
Why Unix commands are short (http://www.catonmat.net/blog/why-unix-commands-are-short/)
OPNsense 17.1.5 released (https://opnsense.org/opnsense-17-1-5-released/)
Something for Apple dual-GPU users (http://lists.dragonflybsd.org/pipermail/commits/2017-April/625847.html)
pkgsrcCon 2017 CFT (https://mail-index.netbsd.org/netbsd-advocacy/2017/05/01/msg000735.html)
TrueOS/Lumina Dev Q&A: May 5th 2017 (https://discourse.trueos.org/t/trueos-lumina-dev-q-a-5-4-17/1347)
Feedback/Questions
Peter - Jails (http://dpaste.com/0J14HGJ#wrap)
Andrew - Languages and University Courses (http://dpaste.com/31AVFSF#wrap)
JuniorJobs (https://wiki.freebsd.org/JuniorJobs)
Steve - TrueOS and Bootloader (http://dpaste.com/1BXVZSY#wrap)
Ben - ZFS questions (http://dpaste.com/0R7AW2T#wrap)
Steve - Linux Emulation (http://dpaste.com/3ZR7NCC#wrap)
| |||
| 15 Aug 2019 | 311: Conference Gear Breakdown | 01:13:25 | |
NetBSD 9.0 release process has started, xargs, a tale of two spellcheckers, Adapting TriforceAFL for NetBSD, Exploiting a no-name freebsd kernel vulnerability, and more.
Headlines
NetBSD 9.0 release process has started (https://mail-index.netbsd.org/netbsd-announce/2019/07/31/msg000301.html)
If you have been following source-changes, you may have noticed the creation of the netbsd-9 branch! It has some really exciting items that we worked on:
+ New AArch64 architecture support:
+ Symmetric and asymmetrical multiprocessing support (aka big.LITTLE)
+ Support for running 32-bit binaries
+ UEFI and ACPI support
+ Support for SBSA/SBBR (server-class) hardware.
+ The FDT-ization of many ARM boards:
+ the 32-bit GENERIC kernel lists 129 different DTS configurations
+ the 64-bit GENERIC64 kernel lists 74 different DTS configurations
+ All supported by a single kernel, without requiring per-board configuration.
+ Graphics driver update, matching Linux 4.4, adding support for up to Kaby Lake based Intel graphics devices.
+ ZFS has been updated to a modern version and seen many bugfixes.
+ New hardware-accelerated virtualization via NVMM.
+ NPF performance improvements and bug fixes. A new lookup algorithm, thmap, is now the default.
+ NVMe performance improvements
+ Optional kernel ASLR support, and partial kernel ASLR for the default configuration.
+ Kernel sanitizers:
+ KLEAK, detecting memory leaks
+ KASAN, detecting memory overruns
+ KUBSAN, detecting undefined behaviour
+ These have been used together with continuous fuzzing via the syzkaller project to find many bugs that were fixed.
+ The removal of outdated networking components such as ISDN and all of its drivers
+ The installer is now capable of performing GPT UEFI installations.
+ Dramatically improved support for userland sanitizers, as well as the option to build all of NetBSD's userland using them for bug-finding.
+ Update to graphics userland: Mesa was updated to 18.3.4, and llvmpipe is now available for several architectures, providing 3D graphics even in the absence of a supported GPU.
We try to test NetBSD as best as we can, but your testing can help NetBSD 9.0 a great release. Please test it and let us know of any bugs you find.
+ Binaries are available at https://nycdn.netbsd.org/pub/NetBSD-daily/netbsd-9/latest/
xargs wtf (https://medium.com/@aarontharris/xargs-wtf-34d2618286b7)
xargs is probably one of the more difficult to understand of the unix command arsenal and of course that just means it’s one of the most useful too.
I discovered a handy trick that I thought was worth a share. Please note there are probably other (better) ways to do this but I did my stackoverflow research and found nothing better.
xargs — at least how I’ve most utilized it — is handy for taking some number of lines as input and doing some work per line. It’s hard to be more specific than that as it does so much else.
It literally took me an hour of piecing together random man pages + tips from 11 year olds on stack overflow, but eventually I produced this gem:
This is an example of how to find files matching a certain pattern and rename each of them. It sounds so trivial (and it is) but it demonstrates some cool tricks in an easy concept.
News Roundup
PkgSrc: A Tale of Two Spellcheckers (https://bentsukun.ch/posts/pkgsrccon-2019/)
This is a transcript of the talk I gave at pkgsrcCon 2019 in Cambridge, UK. It is about spellcheckers, but there are much more general software engineering lessons that we can learn from this case study.
The reason I got into this subject at all was my paternal leave last year, when I finally had some more time to spend working on pkgsrc. It was a tiny item in the enormous TODO file at the top of the source tree (“update enchant to version 2.2”) that made me go into this rabbit hole.
Adapting TriforceAFL for NetBSD, Part 2 (https://blog.netbsd.org/tnf/entry/adapting_triforceafl_for_netbsd_part1)
I have been working on adapting TriforceAFL for NetBSD kernel syscall fuzzing. This blog post summarizes the work done until the second evaluation.
For work done during the first coding period, check out this post.
Summary
> So far, the TriforceNetBSDSyscallFuzzer has been made available in the form of a pkgsrc package with the ability to fuzz most of NetBSD syscalls. In the final coding period of GSoC. I plan to analyse the crashes that were found until now. Integrate sanitizers, try and find more bugs and finally wrap up neatly with detailed documentation.
> Last but not least, I would like to thank my mentor, Kamil Rytarowski for helping me through the process and guiding me. It has been a wonderful learning experience so far!
Exploiting a no-name freebsd kernel vulnerability (https://www.synacktiv.com/posts/exploit/exploiting-a-no-name-freebsd-kernel-vulnerability.html)
A new patch has been recently shipped in FreeBSD kernels to fix a vulnerability (cve-2019-5602) present in the cdrom device. In this post, we will introduce the bug and discuss its exploitation on pre/post-SMEP FreeBSD revisions.
> A closer look at the commit 6bcf6e3 shows that when invoking the CDIOCREADSUBCHANNEL_SYSSPACE ioctl, data are copied with bcopy instead of the copyout primitive. This endows a local attacker belonging to the operator group with an arbitrary write primitive in the kernel memory.
[Allan and Benedicts Conference Gear Breakdown]
Benedict’s Gear:
GlocalMe G3 Mobile Travel HotSpot and Powerbank (https://www.glocalme.com/CA/en-US/cloudsim/g3)
Mogics Power Bagel (http://www.mogics.com/3824-2)
Charby Sense Power Cable (https://charbycharge.com/charby-sense-worlds-smartest-auto-cutoff-cable/)
Allan’s Gear:
Huawei E5770s-320 4G LTE 150 Mbps Mobile WiFi Pro (https://smile.amazon.com/gp/product/B013CEGGKI/)
AOW Global Data SIM Card for On-Demand 4G LTE Mobile Data in Over 90 Countries (https://smile.amazon.com/dp/B071HJFX27/)
All my devices charge from USB-C, so that is great
More USB thumb drives than strictly necessary
My Lenovo X270 laptop running FreeBSD 13-current
My 2016 Macbook Pro (a prize from the raffle at vBSDCon 2017) that I use for email and video conferencing to preserve battery on my FreeBSD machine for work
Beastie Bits
Replacing the Unix tradition (Warning may be rage inducing) (https://www.youtube.com/watch?v=L9v4Mg8wi4U&feature=youtu.be)
Installing OpenBSD over remote serial on the AtomicPI (https://www.thanassis.space/remoteserial.html#remoteserial)
Zen 2 and DragonFly (https://www.dragonflydigest.com/2019/08/05/23294.html)
Improve Docking on FreeBSD (https://blog.yukiisbo.red/posts/2019/05/improve-docking-on-freebsd/)
Register for vBSDCon 2019, Sept 5-7 in Reston VA. Early bird ends August 15th. (https://vbsdcon.com/registration)
Register for EuroBSDCon 2019, Sept 19-22 in Lillehammer, Norway (https://2019.eurobsdcon.org/registration/)
Feedback/Questions
JT - Congrats (http://dpaste.com/0D7Y31E#wrap)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 11 Nov 2021 | 428: Cult of BSD | 00:54:21 | |
OpenBSD Part 1: How it all started, Explaining top(1) on FreeBSD, Measuring power efficiency of a CPU frequency scheduler on OpenBSD, CultBSD, a whole lot of BSD bits, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
What every IT person needs to know about OpenBSD Part 1: How it all started (https://blog.apnic.net/2021/10/28/openbsd-part-1-how-it-all-started/)
Explaining top(1) on FreeBSD (https://klarasystems.com/articles/explaining-top1-on-freebsd/)
News Roundup
Measuring power efficiency of a CPU frequency scheduler on OpenBSD (https://dataswamp.org/~solene/2021-09-26-openbsd-power-usage.html)
CultBSD (https://sourceforge.net/projects/cult-bsd/)
Beastie Bits
• [OpenBSD on the HiFive Unmatched](https://kernelpanic.life/hardware/hifive-unmatched.html)
• [Advanced Documentation Retrieval on FreeBSD](https://adventurist.me/posts/00306)
• [OpenBSD Webzine Issue 3 is out](https://webzine.puffy.cafe/issue-3.html)
• [How to connect and use Bluetooth headphones on FreeBSD](https://forums.freebsd.org/threads/bluetooth-audio-how-to-connect-and-use-bluetooth-headphones-on-freebsd.82671/)
• [How To: Execute Firefox in a jail using iocage and ssh/jailme](https://forums.freebsd.org/threads/how-to-execute-firefox-in-a-jail-using-iocage-and-ssh-jailme.53362/)
• [Understanding AWK](https://earthly.dev/blog/awk-examples/)
• [“Domesticate Your Badgers” Kickstarter Opens](https://mwl.io/archives/13297)
• [Bootstrap an OPNsense development environment in Vagrant](https://github.com/punktDe/vagrant-opnsense)
• [VLANs Bridges and LAG Interface best practice questions](https://www.truenas.com/community/threads/vlans-bridges-and-lag-interface-best-practice-questions.93275/)
• [A Console Desktop](https://pspodcasting.net/dan/blog/2018/console_desktop.html)
• [CharmBUG Casual BSD Meetup and Games (Online)](https://www.meetup.com/CharmBUG/events/281822524)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Dan - ZFS question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/428/feedback/Dan%20-%20ZFS%20question.md)
Lars - Thanks for the interview (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/428/feedback/Lars%20-%20Thanks%20for%20the%20interview.md)
jesse - migrating data from old laptop (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/428/feedback/jesse%20-%20migrating%20data%20from%20old%20laptop.md)
***
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 27 Sep 2017 | 213: The French CONnection | 01:31:00 | |
We recap EuroBSDcon in Paris, tell the story behind a pf PR, and show you how to do screencasting with OpenBSD.
This episode was brought to you by
Headlines
Recap of EuroBSDcon 2017 in Paris, France (https://2017.eurobsdcon.org)
EuroBSDcon was held in Paris, France this year, which drew record numbers this year.
With over 300 attendees, it was the largest BSD event I have ever attended, and I was encouraged by the higher than expected number of first time attendees.
The FreeBSD Foundation held a board meeting on Wednesday afternoon with the members who were in Paris. Topics included future conferences (including a conference kit we can mail to people who want to represent FreeBSD) and planning for next year.
The FreeBSD Devsummit started on Thursday at the beautiful Mozilla Office in Paris. After registering and picking up our conference bag, everyone gathered for a morning coffee with lots of handshaking and greeting. We then gathered in the next room which had a podium with microphone, screens as well as tables and chairs. After developers sat down, Benedict opened the devsummit with a small quiz about France for developers to win a Mogics Power Bagel (https://www.mogics.com/?page_id=3824). 45 developers participated and DES won the item in the end. After introductions and collecting topics of interest from everyone, we started with the Work in Progress (WIP) session.
The WIP session had different people present a topic they are working on in 7 minute timeslots. Topics ranged from FreeBSD Forwarding Performance, fast booting options, and a GELI patch under review to attach multiple providers. See their slides on the FreeBSD wiki (https://wiki.freebsd.org/DevSummit/201709).
After lunch, the FreeBSD Foundation gave a general update on staff and funding, as well as a more focused presentation about our partnership with Intel. People were interested to hear what was done so far and asked a few questions to the Intel representative Glenn Weinberg.
After lunch, developers worked quietly on their own projects. The mic remained open and occasionally, people would step forward and gave a short talk without slides or motivated a discussion of common interest. The day concluded with a dinner at a nice restaurant in Paris, which allowed to continue the discussions of the day.
The second day of the devsummit began with a talk about the CAM-based SDIO stack by Ilya Bakulin. His work would allow access to wifi cards/modules on embedded boards like the Raspberry Pi Zero W and similar devices as many of these are using SDIO for data transfers.
Next up was a discussion and Q&A session with the FreeBSD core team members who were there (missing only Benno Rice, Kris Moore, John Baldwin, and Baptiste Daroussin, the latter being busy with conference preparations). The new FCP (FreeBSD community proposals) were introduced for those who were not at BSDCan this year and the hows and whys about it. Allan and I were asked to describe our experiences as new members of core and we encouraged people to run for core when the next election happens. After a short break, Scott Long gave an overview of the work that’s been started on NUMA (Non-Uniform Memory Architecture), what the goals of the project are and who is working on it.
Before lunch, Christian Schwarz presented his work on zrepl, a new ZFS replication solution he developed using Go. This sparked interest in developers, a port was started (https://reviews.freebsd.org/D12462) and people suggested to Christian that he should submit his talk to AsiaBSDcon and BSDCan next year. Benedict had to leave before lunch was done to teach his Ansible tutorial (which was well attended) at the conference venue.
There were organized dinners, for those two nights, quite a feat of organization to fit over 100 people into a restaurant and serve them quickly.
On Saturday, there was a social event, a river cruise down the Seine. This took the form of a ‘standing’ dinner, with a wide selection of appetizer type dishes, designed to get people to walk around and converse with many different people, rather than sit at a table with the same 6-8 people. I talked to a much larger group of people than I had managed to at the other dinners.
I like having both dinner formats.
We would also like to thank all of the BSDNow viewers who attended the conference and made the point of introducing themselves to us. It was nice to meet you all.
The recordings of the live video stream from the conference are available immediately, so you can watch the raw versions of the talks now:
Auditorium
Keynote 1: Software Development in the Age of Heroes (https://youtu.be/4iR8g9-39LM?t=179) by Thomas Pornin (https://twitter.com/BearSSLnews)
Tuning FreeBSD for routing and firewalling (https://youtu.be/4iR8g9-39LM?t=1660) by Olivier Cochard-Labbé (https://twitter.com/ocochardlabbe)
My BSD sucks less than yours, Act I (https://youtu.be/4iR8g9-39LM?t=7040) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_)
My BSD sucks less than yours, Act II (https://youtu.be/4iR8g9-39LM?t=14254) by Antoine Jacoutot (https://twitter.com/ajacoutot) and Baptiste Daroussin (https://twitter.com/_bapt_)
Reproducible builds on NetBSD (https://youtu.be/4iR8g9-39LM?t=23351) by Christos Zoulas
Your scheduler is not the problem (https://youtu.be/4iR8g9-39LM?t=26845) by Martin Pieuchot
Keynote 2: A French story on cybercrime (https://youtu.be/4iR8g9-39LM?t=30540) by Éric Freyssinet (https://twitter.com/ericfreyss)
Case studies of sandboxing base system with Capsicum (https://youtu.be/jqdHYEH_BQY?t=731) by Mariusz Zaborski (https://twitter.com/oshogbovx)
OpenBSD’s small steps towards DTrace (a tale about DDB and CTF) (https://youtu.be/jqdHYEH_BQY?t=6030) by Jasper Lievisse Adriaanse
The Realities of DTrace on FreeBSD (https://youtu.be/jqdHYEH_BQY?t=13096) by George Neville-Neil (https://twitter.com/gvnn3)
OpenSMTPD, current state of affairs (https://youtu.be/jqdHYEH_BQY?t=16818) by Gilles Chehade (https://twitter.com/PoolpOrg)
Hoisting: lessons learned integrating pledge into 500 programs (https://youtu.be/jqdHYEH_BQY?t=21764) by Theo de Raadt
Keynote 3: System Performance Analysis Methodologies (https://youtu.be/jqdHYEH_BQY?t=25463) by Brendan Gregg (https://twitter.com/brendangregg)
Closing Session (https://youtu.be/jqdHYEH_BQY?t=29355)
Karnak
“Is it done yet ?” The never ending story of pkg tools (https://youtu.be/1hjzleqGRYk?t=71) by Marc Espie (https://twitter.com/espie_openbsd)
A Tale of six motherboards, three BSDs and coreboot (https://youtu.be/1hjzleqGRYk?t=7498) by Piotr Kubaj and Katarzyna Kubaj
State of the DragonFly’s graphics stack (https://youtu.be/1hjzleqGRYk?t=11475) by François Tigeot
From NanoBSD to ZFS and Jails – FreeBSD as a Hosting Platform, Revisited (https://youtu.be/1hjzleqGRYk?t=16227) by Patrick M. Hausen
Bacula – nobody ever regretted making a backup (https://youtu.be/1hjzleqGRYk?t=20069) by Dan Langille (https://twitter.com/DLangille)
Never Lose a Syslog Message (https://youtu.be/qX0BS4P65cQ?t=325) by Alexander Bluhm
Running CloudABI applications on a FreeBSD-based Kubernetes cluster (https://youtu.be/qX0BS4P65cQ?t=5647) by Ed Schouten (https://twitter.com/EdSchouten)
The OpenBSD web stack (https://youtu.be/qX0BS4P65cQ?t=13255) by Michael W. Lucas (https://twitter.com/mwlauthor)
The LLDB Debugger on NetBSD (https://youtu.be/qX0BS4P65cQ?t=16835) by Kamil Rytarowski
What’s in store for NetBSD 8.0? (https://youtu.be/qX0BS4P65cQ?t=21583) by Alistair Crooks
Louxor
A Modern Replacement for BSD spell(1) (https://youtu.be/6Nen6a1Xl7I?t=156) by Abhinav Upadhyay (https://twitter.com/abhi9u)
Portable Hotplugging: NetBSD’s uvm_hotplug(9) API development (https://youtu.be/6Nen6a1Xl7I?t=5874) by Cherry G. Mathew
Hardening pkgsrc (https://youtu.be/6Nen6a1Xl7I?t=9343) by Pierre Pronchery (https://twitter.com/khorben)
Discovering OpenBSD on AWS (https://youtu.be/6Nen6a1Xl7I?t=14874) by Laurent Bernaille (https://twitter.com/lbernail)
OpenBSD Testing Infrastructure Behind bluhm.genua.de (https://youtu.be/6Nen6a1Xl7I?t=18639) by Jan Klemkow
The school of hard knocks – PT1 (https://youtu.be/8wuW8lfsVGc?t=276) by Sevan Janiyan (https://twitter.com/sevanjaniyan)
7 years of maintaining firefox, and still looking ahead (https://youtu.be/8wuW8lfsVGc?t=5321) by Landry Breuil
Branch VPN solution based on OpenBSD, OSPF, RDomains and Ansible (https://youtu.be/8wuW8lfsVGc?t=12385) by Remi Locherer
Running BSD on AWS (https://youtu.be/8wuW8lfsVGc?t=15983) by Julien Simon and Nicolas David
Getting started with OpenBSD device driver development (https://youtu.be/8wuW8lfsVGc?t=21491) by Stefan Sperling
A huge thanks to the organizers, program committee, and sponsors of EuroBSDCon. Next year, EuroBSDcon will be in Bucharest, Romania.
***
The story of PR 219251 (https://www.sigsegv.be//blog/freebsd/PR219251)
The actual story I wanted Kristof to tell, the pf bug he fixed at the Essen Hackathon earlier this summer.
As I threatened to do in my previous post, I'm going to talk about PR 219251 for a bit. The bug report dates from only a few months ago, but the first report (that I can remeber) actually came from Shawn Webb on Twitter, of all places
Despite there being a stacktrace it took quite a while (nearly 6 months in fact) before I figured this one out.
It took Reshad Patuck managing to distill the problem down to a small-ish test script to make real progress on this. His testcase meant that I could get core dumps and experiment. It also provided valuable clues because it could be tweaked to see what elements were required to trigger the panic.
This test script starts a (vnet) jail, adds an epair interface to it, sets up pf in the jail, and then reloads the pf rules on the host. Interestingly the panic does not seem to occur if that last step is not included.
Obviously not the desired behaviour, but it seems strange. The instances of pf in the jails are supposed to be separate.
We try to fetch a counter value here, but instead we dereference a bad pointer. There's two here, so already we need more information. Inspection of the core dump reveals that the state pointer is valid, and contains sane information. The rule pointer (rule.ptr) points to a sensible location, but the data is mostly 0xdeadc0de. This is the memory allocator being helpful (in debug mode) and writing garbage over freed memory, to make use-after-free bugs like this one easier to find.
In other words: the rule has been free()d while there was still a state pointing to it. Somehow we have a state (describing a connection pf knows about) which points to a rule which no longer exists. The core dump also shows that the problem always occurs with states and rules in the default vnet (i.e. the host pf instance), not one of the pf instances in one of the vnet jails. That matches with the observation that the test script does not trigger the panic unless we also reload the rules on the host.
Great, we know what's wrong, but now we need to work out how we can get into this state. At this point we're going to have to learn something about how rules and states get cleaned up in pf. Don't worry if you had no idea, because before this bug I didn't either.
The states keep a pointer to the rule they match, so when rules are changed (or removed) we can't just delete them. States get cleaned up when connections are closed or they time out. This means we have to keep old rules around until the states that use them expire.
When rules are removed pfunlinkrule() adds then to the Vpfunlinkedrules list (more on that funny V prefix later). From time to time the pf purge thread will run over all states and mark the rules that are used by a state. Once that's done for all states we know that all rules that are not marked as in-use can be removed (because none of the states use it). That can be a lot of work if we've got a lot of states, so pfpurgethread() breaks that up into smaller chuncks, iterating only part of the state table on every run.
We iterate over all of our virtual pf instances (VNETFOREACH()), check if it's active (for FreeBSD-EN-17.08, where we've seen this code before) and then check the expired states with pfpurgeexpiredstates(). We start at state 'idx' and only process a certain number (determined by the PFTMINTERVAL setting) states. The pfpurgeexpiredstates() function returns a new idx value to tell us how far we got.
So, remember when I mentioned the odd V_ prefix? Those are per-vnet variables. They work a bit like thread-local variables. Each vnet (virtual network stack) keeps its state separate from the others, and the V_ variables use a pointer that's changed whenever we change the currently active vnet (say with CURVNETSET() or CURVNETRESTORE()). That's tracked in the 'curvnet' variable. In other words: there are as many Vpfvnetactive variables as there are vnets: number of vnet jails plus one (for the host system).
Why is that relevant here? Note that idx is not a per-vnet variable, but we handle multiple pf instances here. We run through all of them in fact. That means that we end up checking the first X states in the first vnet, then check the second X states in the second vnet, the third X states in the third and so on and so on.
That of course means that we think we've run through all of the states in a vnet while we really only checked some of them. So when pfpurgeunlinkedrules() runs it can end up free()ing rules that actually are still in use because pfpurgethread() skipped over the state(s) that actually used the rule. The problem only happened if we reloaded rules in the host, because the active ruleset is never free()d, even if there are no states pointing to the rule.
That explains the panic, and the fix is actually quite straightforward: idx needs to be a per-vnet variable, Vpfpurge_idx, and then the problem is gone. As is often the case, the solution to a fairly hard problem turns out to be really simple.
As you might expect, finding the problem takes a lot more work that fixing it
Thanks to Kristof for writing up this detailed post explaining how the problem was found, and what caused it.
***
vBSDcon 2017: BSD at Work (https://www.ixsystems.com/blog/vbsdcon-2017-dexter/)
The third biennial vBSDcon hosted by Verisign took place September 7th through 9th with the FreeBSD Developer Summit taking place the first day. vBSDcon and iXsystems’ MeetBSD event have been alternating between the East and West coasts of the U.S.A. and these two events play vital roles in reaching Washington, DC-area and Bay Area/Silicon Valley audiences. Where MeetBSD serves many BSD Vendors, vBSDcon attracts a unique government and security industry demographic that isn’t found anywhere else. Conference time and travel budgets are always limited and bringing these events to their attendees is a much-appreciated service provided by their hosts.
The vBSDcon FreeBSD DevSummit had a strong focus on OpenZFS, the build system and networking with the FreeBSD 12 wish list of features in mind. How to best incorporate the steady flow of new OpenZFS features into FreeBSD such as dataset-level encryption was of particular interest. This feature from a GNU/Linux-based storage vendor is tribute to the growth of the OpenZFS community which is vital in light of the recent “Death of Solaris and ZFS” at Oracle. There has never been more demand for OpenZFS on FreeBSD and the Oracle news further confirms our collective responsibility to meet that demand.
The official conference opened with my talk on “Isolated BSD Build Environments” in which I explained how the bhyve hypervisor can be used to effortlessly tour FreeBSD 5.0-onward and build specific source releases on demand to trace regressions to their offending commit. I was followed by a FreeNAS user who made the good point that FreeNAS is an exemplary “entry vector” into Unix and Enterprise Storage fundamentals, given that many of the vectors our generation had are gone. Where many of us discovered Unix and the Internet via console terminals at school or work, smart phones are only delivering the Internet without the Unix. With some irony, both iOS and Android are Unix-based yet offer few opportunities for their users to learn and leverage their Unix environments.
The next two talks were The History and Future of Core Dumps in FreeBSD by Sam Gwydir and Using pkgsrc for multi-platform deployments in heterogeneous environments by G. Clifford Williams. I strongly recommend that anyone wanting to speak at AsiaBSDCon read Sam’s accompanying paper on core dumps because I consider it the perfect AsiaBSDCon topic and his execution is excellent. Core dumps are one of those things you rarely think about until they are a DROP EVERYTHING! priority. G. Clifford’s talk was about what I consider a near-perfect BSD project: pkgsrc, the portable BSD package manager. I put it up there with OpenSSH and mandoc as projects that have provided significant value to other Open Source operating systems. G. Clifford’s real-world experiences are perfectly inline with vBSDcon’s goal to be more production-oriented than other BSDCons.
Of the other talks, any and all Dtrace talks are always appreciated and George Neville-Neil’s did not disappoint. He based it on his experiences with the Teach BSD project which is bringing FreeBSD-based computer science education to schools around the world. The security-related talks by John-Mark Gurney, Dean Freeman and Michael Shirk also represented vBSDcon’s consideration of the local community and made a convincing point that the BSDs should make concerted efforts to qualify for Common Criteria, FIPS, and other Government security requirements. While some security experts will scoff at these, they are critical to the adoption of BSD-based products by government agencies.
BSD Now hosts Allan Jude and Benedict Reuschling hosted an OpenZFS BoF and Ansible talk respectively and I hosted a bhyve hypervisor BoF. The Hallway Track and food at vBSDcon were excellent and both culminated with an after-dinner dramatic reading of Michael W. Lucas’ latest book that raised money for the FreeBSD Foundation. A great time was had by all and it was wonderful to see everyone!
News Roundup
FreeBSD 10.4-RC2 Available (https://lists.freebsd.org/pipermail/freebsd-stable/2017-September/087848.html)
FreeBSD 10.4 will be released soon, this is the last chance to find bugs before the official release is cut.
Noteworthy Changes Since 10.4-RC1:
Given that the amd64 disc1 image was overflowing, more of the base components installed into the disc1 (live) file systems had to be disabled. Most notably, this removed the compiler toolchain from the disc1 images. All disabled tools are still available with the dvd1 images, though.
The aesni(4) driver now no longer shares a single FPU context across multiple sessions in multiple threads, addressing problems seen when employing aesni(4) for ipsec(4).
Support for netmap(4) by the ixgbe(4) driver has been brought into line with the netmap(4) API present in stable/10. Also, ixgbe(4) now correctly handles VFs in its netmap(4) support again instead of treating these as PFs.
During the creation of amd64 and i386 VM images, etcupdate(8) and mergemaster(8) databases now are bootstrapped, akin to what happens along the extraction of base.txz as part of a new installation via bsdinstall(8). This change allows for both of these tools to work out-of-box on the VM images and avoids errors seen when upgrading these images via freebsd-update(8).
If you are still on the stable/10 branch, you should test upgrading to 10.4, and make sure there are no problems with your workload
Additional testing specifically of the features that have changed since 10.4-BETA1 would also be most helpful
This will be the last release from the stable/10 branch
***
OpenBSD changes of note 628 (https://www.tedunangst.com/flak/post/openbsd-changes-of-note-628)
EuroBSDCon in two weeks. Be sure to attend early and often.
Many and various documentation improvements for libcrypto. New man pages, rewrites, expanded bugs sections, and more.
Only allow upward migration in vmd.
There’s a README for the syspatch build system if you want to run your own.
Move the kernel relinking code from /etc/rc into a seperate script usable by syspatch. Kernel patches can now be reduced to just the necessary files.
Make the callers of sogetopt() responsible for allocating memory. Now allocation and free occur in the same place.
Use waitpid() instead of wait() in most programs to avoid accidentally collecting the wrong child.
Have cu call isatty() before making assumptions.
Switch mandoc rendering of mathematical symbols and greek letters from trying to imitate the characters’ graphical shapes, which resulted in unintelligible renderings in many cases, to transliterations conveying the characters’ meanings.
Update libexpat to 2.2.4. Fix copying partial UTF-8 characters.
Sigh, here we go again.
Work around bug in F5’s handling of the supported elliptic curves extension. RFC 4492 only defines elliptic_curves for ClientHello. However, F5 is sending it in ServerHello. We need to skip over it since our TLS extension parsing code is now more strict.
After a first install, run syspatch -c to check for patches.
If SMAP is present, clear PSL_AC on kernel entry and interrupt so that only the code in copy{in,out}* that need it run with it set. Panic if it’s set on entry to trap() or syscall(). Prompted by Maxime Villard’s NetBSD work. Errata.
New drivers for arm: rktemp, mvpinctrl, mvmpic, mvneta, mvmdio, mvpxa, rkiic, rkpmic.
No need to exec rm from within mandoc. We know there’s exactly one file and directory to remove. Similarly with running cmp.
Revert to Mesa 13.0.6 to hopefully address rendering issues a handful of people have reported with xpdf/fvwm on ivy bridge with modesetting driver.
Rewrite ALPN extension using CBB/CBS and the new extension framework. Rewrite SRTP extension using CBB/CBS and the new extension framework.
Revisit 2q queue sizes. Limit the hot queue to 1/20th the cache size up to a max of 4096 pages. Limit the warm and cold queues to half the cache. This allows us to more effectively notice re-interest in buffers instead of losing it in a large hot queue.
Add glass console support for arm64. Probably not yet for your machine, though.
Replace heaps of hand-written syscall stubs in ld.so with a simpler framework.
65535 is a valid port to listen on.
When xinit starts an X server that listens only on UNIX socket, prefer DISPLAY=unix:0 rather than DISPLAY=:0. This will prevent applications from ever falling back to TCP if the UNIX socket connection fails (such as when the X server crashes). Reverted.
Add -z and -Z options to apmd to auto suspend or hibernate when low on battery.
Remove the original (pre-IETF) chacha20-poly1305 cipher suites.
Add urng(4) which supports various USB RNG devices. Instead of adding one driver per device, start bundling them into a single driver.
Remove old deactivated pledge path code. A replacement mechanism is being brewed.
Fix a bug from the extension parsing rewrite. Always parse ALPN even if no callback has been installed to prevent leaving unprocessed data which leads to a decode error.
Clarify what is meant by syslog priorities being ordered, since the numbers and priorities are backwards.
Remove a stray setlocale() from ksh, eliminating a lot of extra statically linked code.
Unremove some NPN symbols from libssl because ports software thinks they should be there for reasons.
Fix saved stack location after resume. Somehow clang changed it. Resume works again on i386.
Improve error messages in vmd and vmctl to be more informative.
Stop building the miniroot installer for OMAP3 Beagleboards. It hasn’t worked in over a year and nobody noticed.
Have the callers of sosetopt() free the mbuf for symmetry.
On octeon, let the kernel use the hardware FPU even if emulation is compiled in. It’s faster.
Fix support for 486DX CPUs by not calling cpuid. I used to own a 486. Now I don’t.
Merge some drm fixes from linux.
Defer probing of floppy drives, eliminating delays during boot.
Better handling of probes and beacons and timeouts and scans in wifi stack to avoid disconnects.
Move mutex, condvar, and thread-specific data routes, pthreadonce, and pthreadexit from libpthread to libc, along with low-level bits to support them. Let’s thread aware (but not actually threaded) code work with just libc.
New POSIX xlocale implementation. Complete as long as you only use ASCII and UTF-8, as you should.
Round and round it goes; when 6.2 stops, nobody knows. A peak at the future?
***
Screencasting with OpenBSD (http://eradman.com/posts/screencasting.html)
USB Audio
Any USB microphone should appear as a new audio device. Here is the dmesg for my mic by ART:
uaudio0 at uhub0 port 2 configuration 1 interface 0 "M-One USB" rev 1.10/0.01 addr 2
uaudio0: audio rev 1.00, 8 mixer controls
audio1 at uaudio0
audioctl can read off all of the specific characterisitcs of this device
$ audioctl -f /dev/audio1 | grep record
mode=play,record
record.rate=48000
record.channels=1
record.precision=16
record.bps=2
record.msb=1
record.encoding=slinear_le
record.pause=0
record.active=0
record.block_size=1960
record.bytes=0
record.errors=0
Now test the recording from the second audio device using aucat(1)
aucat -f rsnd/1 -o file.wav
If the device also has a headset audio can be played through the same device.
aucat -f rsnd/1 -i file.wav
Screen Capture using Xvfb
The rate at which a framebuffer for your video card is a feature of the hardware and software your using, and it's often very slow. x11vnc will print an estimate of the banwidth for the system your running.
x11vnc
...
09/05/2012 22:23:45 fb read rate: 7 MB/sec
This is about 4fps. We can do much better by using a virtual framebuffer. Here I'm setting up a new screen, setting the background color, starting cwm and an instance of xterm
Xvfb :1 -screen 0 720x540x16 &
DISPLAY=:1 xsetroot -solid steelblue &
DISPLAY=:1 cwm &
DISPLAY=:1 xterm +sb -fa Hermit -fs 14 &
Much better! Now we're up around 20fps.
x11vnc -display :1 &
...
11/05/2012 18:04:07 fb read rate: 168 MB/sec
Make a connection to this virtual screen using raw encoding to eliminate time wasted on compression.
vncviewer localhost -encodings raw
A test recording with sound then looks like this
ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 ~/out.avi
Note: always stop the recording and playback using q, not Ctrl-C so that audio inputs are shut down properly.
Screen Capture using Xephyr
Xephyr is perhaps the easiest way to run X with a shadow framebuffer. This solution also avoids reading from the video card's RAM, so it's reasonably fast.
Xephyr -ac -br -noreset -screen 800x600 :1 &
DISPLAY=:1 xsetroot -solid steelblue &
DISPLAY=:1 cwm &
DISPLAY=:1 xrdb -load ~/.Xdefaults &
DISPLAY=:1 xterm +sb -fa "Hermit" -fs 14 &
Capture works in exactally the same way. This command tries to maintain 12fps.
ffmpeg -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi
To capture keyboard and mouse input press Ctrl then Shift. This is very handy for using navigating a window manager in the nested X session.
Arranging Windows
I have sometimes found it helpful to launch applications and arrange them in a specific way. This will open up a web browser listing the current directory and position windows using xdotool
DISPLAY=:1 midori "file:///pwd" &
sleep 2
DISPLAY=:1 xdotool search --name "xterm" windowmove 0 0
DISPLAY=:1 xdotool search --class "midori" windowmove 400 0
DISPLAY=:1 xdotool search --class "midori" windowsize 400 576
This will position the window precisely so that it appears to be in a tmux window on the right.
Audio/Video Sync
If you find that the audio is way out of sync with the video, you can ajust the start using the -ss before the audio input to specify the number of seconds to delay. My final recording command line, that delays the audio by 0.5 seconds, writing 12fps
ffmpeg -ss 0.5 -f sndio -i snd/1 -y -f x11grab -r 12 -s 800x600 -i :1.0 -vcodec ffv1 -acodec copy ~/out.avi
Sharing a Terminal with tmux
If you're trying to record a terminal session, tmux is able to share a session. In this way a recording of an X framebuffer can be taken without even using the screen. Start by creating the session.
tmux -2 -S /tmp/tmux0
Then on the remote side connect on the same socket
tmux -2 -S /tmp/tmux0 attach
Taking Screenshots
Grabbing a screenshots on Xvfb server is easily accomplished with ImageMagick's import command
DISPLAY=:1 import -window root screenshot.png
Audio Processing and Video Transcoding
The first step is to ensure that the clip begins and ends where you'd like it to. The following will make a copy of the recording starting at time 00:00 and ending at 09:45
ffmpeg -i interactive-sql.avi \
-vcodec copy -acodec copy
-ss 00:00:00 -t 00:09:45
interactive-sql-trimmed.avi
mv interactive-sql-trimmed.avi interactive-sql.avi
Setting the gain correctly is very important with an analog mixer, but if you're using a USB mic there may not be a gain option; simply record using it's built-in settings and then adjust the levels afterwards using a utility such as normalize. First extact the audio as a raw PCM file and then run normalize
ffmpeg -i interactive-sql.avi -c:a copy -vn audio.wav
normalize audio.wav
Next merge the audio back in again
ffmpeg -i interactive-sql.avi -i audio.wav \
-map 0:0 -map 1:0 -c copy interactive-sql-normalized.avi
The final step is to compress the screencast for distribution. Encoding to VP8/Vorbis is easy:
ffmpeg -i interactive-sql-normalized.avi -c:v libvpx -b:v 1M
-c:a libvorbis -q:a 6 interactive-sql.webm
H.264/AAC is tricky. For most video players the color space needs to be set to yuv420p. The -movflags puts the index data at the beginning of the file to enable streaming/partial content requests over HTTP:
ffmpeg -y -i interactive-sql-normalized.avi -c:v libx264 \
-preset slow -crf 14 -pix_fmt yuv420p -movflags +faststart \
-c:a aac -q:a 6 interactive-sql.mp4
TrueOS @ Ohio Linuxfest ’17! (https://www.trueos.org/blog/trueos-ohio-linuxfest-17/)
Dru Lavigne and Ken Moore are both giving presentations on Saturday the 30th. Sit in and hear about new developments for the Lumina and FreeNAS projects.
Ken is offering Lumina Rising: Challenging Desktop Orthodoxy at 10:15 am in Franklin A. Hear his thoughts about the ideas propelling desktop environment development and how Lumina, especially Lumina 2, is seeking to offer a new model of desktop architecture. Elements discussed include session security, application dependencies, message handling, and operating system integration.
Dru is talking about What’s New in FreeNAS 11 at 2:00 pm in Franklin D. She’ll be providing an overview of some of the new features added in FreeNAS 11.0, including:
Alert Services
Starting specific services at boot time
AD Monitoring to ensure the AD service restarts if disconnected
A preview of the new user interface
support for S3-compatible storage and the bhyve hypervisor
She’s also giving a sneak peek of FreeNAS 11.1, which has some neat features:
A complete rewrite of the Jails/Plugins system as FreeNAS moves from warden to iocage
Writing new plugins with just a few lines of code
A brand new asynchronous middleware API
Who’s going? Attending this year are:
Dru Lavigne (dlavigne): Dru leads the technical documentation team at iX, and contributes heavily to open source documentation projects like FreeBSD, FreeNAS, and TrueOS.
Ken Moore (beanpole134): Ken is the lead developer of Lumina and a core contributor to TrueOS. He also works on a number of other Qt5 projects for iXsystems.
J.T. Pennington (q5sys): Some of you may be familiar with his work on BSDNow, but J.T. also contributes to the TrueOS, Lumina, and SysAdm projects, helping out with development and general bug squashing.
***
Beastie Bits
Lumina Development Preview: Theme Engine (https://www.trueos.org/blog/lumina-development-preview-theme-engine/)
It's happening! Official retro Thinkpad lappy spotted in the wild (https://www.theregister.co.uk/2017/09/04/retro_thinkpad_spotted_in_the_wild/)
LLVM libFuzzer and SafeStack ported to NetBSD (https://blog.netbsd.org/tnf/entry/llvm_libfuzzer_and_safestack_ported)
Remaining 2017 FreeBSD Events (https://www.freebsdfoundation.org/news-and-events/event-calendar/2017-openzfs-developer-summit/)
***
Feedback/Questions
Andrew - BSD Teaching Material (http://dpaste.com/0YTT0VP)
Seth - Switching to Tarsnap after Crashplan becomes no more (http://dpaste.com/1SK92ZX#wrap)
Thomas - Native encryption in ZFS (http://dpaste.com/02KD5FX#wrap)
Coding Cowboy - Coding Cowboy - Passwords and clipboards (http://dpaste.com/31K0E40#wrap)
***
| |||
| 20 Jun 2019 | 303: OpenZFS in Ports | 00:52:33 | |
OpenZFS-kmod port available, using blacklistd with NPF as fail2ban replacement, ZFS raidz expansion alpha preview 1, audio VU-meter increases CO2 footprint rant, XSAVE and compat32 kernel work for LLDB, where icons for modern X applications come from, and more.
Headlines
ZFSonFreeBSD ports renamed OpenZFS (https://www.freshports.org/sysutils/openzfs-kmod)
The ZFS on FreeBSD project has renamed the userland and kernel ports from zol and zol-kmod to openzfs and openzfs-kmod
The new versions from this week are IOCTL compatible with the command line tools in FreeBSD 12.0, so you can use the old userland with the new kernel module (although obviously not the new features)
With the renaming it is easier to specify which kernel module you want to load in /boot/loader.conf:
> zfs_load=”YES”
or
> openzfs_load=”YES”
To load traditional or the newer version of ZFS
The kmod still requires FreeBSD 12-stable or 13-current because it depends on the newer crypto support in the kernel for the ZFS native encryption feature. Allan is looking at ways to work around this, but it may not be practical.
We would like to do an unofficial poll on how people would the userland to co-exist. Add a suffix to the new commands in /usr/local (zfs.new zpool.new or whatever). One idea i’ve had is to move the zfs and zpool commands to /libexec and make /sbin/zfs and /sbin/zpool a switcher script, that will call the base or ports version based on a config file (or just based on if the port is installed)
For testing purposes, generally you should be fine as long as you don’t run ‘zpool upgrade’, which will make your pool only importable using the newer ZFS.
For extra safety, you can create a ‘zpool checkpoint’, which will allow you to undo any changes that are made to the pool during your testing with the new openzfs tools. Note: the checkpoint will undo EVERYTHING. So don’t save new data you want to keep.
Note: Checkpoints disable all freeing operations, to prevent any data from being overwritten so that you can re-import at the checkpoint and undo any operation (including zfs destroy-ing a dataset), so also be careful you don’t run out of space during testing.
Please test and provide feedback.
How to use blacklistd(8) with NPF as a fail2ban replacement (https://www.unitedbsd.com/d/63-how-to-use-blacklistd8-with-npf-as-a-fail2ban-replacement)
About blacklistd(8)
blacklistd(8) provides an API that can be used by network daemons to communicate with a packet filter via a daemon to enforce opening and closing ports dynamically based on policy.
The interface to the packet filter is in /libexec/blacklistd-helper (this is currently designed for npf) and the configuration file (inspired from inetd.conf) is in etc/blacklistd.conf
Now, blacklistd(8) will require bpfjit(4) (Just-In-Time compiler for Berkeley Packet Filter) in order to properly work, in addition to, naturally, npf(7) as frontend and syslogd(8), as a backend to print diagnostic messages. Also remember npf shall rely on the npflog* virtual network interface to provide logging for tcpdump() to use.
Unfortunately (dont' ask me why :P) in 8.1 all the required kernel components are still not compiled by default in the GENERIC kernel (though they are in HEAD), and are rather provided as modules. Enabling NPF and blacklistd services would normally result in them being automatically loaded as root, but predictably on securelevel=1 this is not going to happen
News Roundup
[WIP] raidz expansion, alpha preview 1 (https://github.com/zfsonlinux/zfs/pull/8853)
Motivation and Context
> This is a alpha-quality preview of RAID-Z expansion. This feature allows disks to be added one at a time to a RAID-Z group, expanding its capacity incrementally. This feature is especially useful for small pools (typically with only one RAID-Z group), where there isn't sufficient hardware to add capacity by adding a whole new RAID-Z group (typically doubling the number of disks).
> For additional context as well as a design overview, see my short talk from the 2017 OpenZFS Developer Summit: slides video
Rant: running audio VU-meter increases my CO2 footprint (https://medium.com/@MartinCracauer/bug-rant-running-audio-vu-meter-increases-my-co2-footprint-871d5c1bee5a)
A couple months ago I noticed that the monitor on my workstation never power off anymore. Screensaver would go on, but DPMs (to do the poweroff) never kicked in.
I grovels the output of various tools that display DPMS settings, which as usual in Xorg were useless. Everybody said DPMS is on with a timeout. I even wrote my own C program to use every available Xlib API call and even the xscreensaver library calls. (should make it available) No go, everybody says that DPMs is on, enabled and set on a timeout. Didn’t matter whether I let xscreeensaver do the job or just the X11 server.
After a while I noticed that DPMS actually worked between starting my X11 server and starting all my clients. I have a minimal .xinitrc and start the actual session from a script, that is how I could notice. If I used a regular desktop login I wouldn’t have noticed. A server state bug was much more likely than a client bug.
+ See the article for the rest...
XSAVE and compat32 kernel work for LLDB (http://blog.netbsd.org/tnf/entry/xsave_and_compat32_kernel_work)
Upstream describes LLDB as a next generation, high-performance debugger. It is built on top of LLVM/Clang toolchain, and features great integration with it. At the moment, it primarily supports debugging C, C++ and ObjC code, and there is interest in extending it to more languages.
In February, I have started working on LLDB, as contracted by the NetBSD Foundation. So far I've been working on reenabling continuous integration, squashing bugs, improving NetBSD core file support and lately extending NetBSD's ptrace interface to cover more register types. You can read more about that in my Apr 2019 report.
In May, I was primarily continuing the work on new ptrace interface. Besides that, I've found and fixed a bug in ptrace() compat32 code, pushed LLVM buildbot to ‘green’ status and found some upstream LLVM regressions. More below.
Some things about where icons for modern X applications come from (https://utcc.utoronto.ca/~cks/space/blog/unix/ModernXAppIcons)
If you have a traditional window manager like fvwm, one of the things it can do is iconify X windows so that they turn into icons on the root window (which would often be called the 'desktop'). Even modern desktop environments that don't iconify programs to the root window (or their desktop) may have per-program icons for running programs in their dock or taskbar. If your window manager or desktop environment can do this, you might reasonably wonder where those icons come from by default.
Although I don't know how it was done in the early days of X, the modern standard for this is part of the Extended Window Manager Hints. In EWMH, applications give the window manager a number of possible icons, generally in different sizes, as ARGB bitmaps (instead of, say, SVG format). The window manager or desktop environment can then pick whichever icon size it likes best, taking into account things like the display resolution and so on, and display it however it wants to (in its original size or scaled up or down).
How this is communicated in specific is through the only good interprocess communication method that X supplies, namely X properties. In the specific case of icons, the NETWMICON property is what is used, and xprop can display the size information and an ASCII art summary of what each icon looks like. It's also possible to use some additional magic to read out the raw data from _NETWM_ICON in a useful format; see, for example, this Stackoverflow question and its answers.
Beastie Bits
Recent Security Innovations (http://undeadly.org/cgi?action=article;sid=20190605110020)
Old Unix books + Solaris (https://imgur.com/a/HbSYtQI)
Pro-Desktop - A Tiling Desktop Environment (https://bitcannon.net/post/pro-desktop/)
The Tar Pipe (https://blog.extracheese.org/2010/05/the-tar-pipe.html)
At least one vim trick you might not know (https://www.hillelwayne.com/post/intermediate-vim/)
Feedback/Questions
Johnny - listener feedback (http://dpaste.com/0ZQCQ8Y#wrap)
Brian - Questions (http://dpaste.com/1843RNX#wrap)
Mark - ZFS Question (http://dpaste.com/3M83X9G#wrap)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 30 Nov 2016 | 170: Sandboxing Cohabitation | 01:16:24 | |
This week on the show, we’ve got some new info on the talks from EuroBSDCon, a look at sharing a single ZFS pool between Linux and BSD, Sandboxing and much more! Stay tuned for your place to B...SD!
This episode was brought to you by
Headlines
EuroBSDcon 2016 Presentation Slides (https://2016.eurobsdcon.org/PresentationSlides/)
Due to circumstances beyond the control of the organizers of EuroBSDCon, there were not recordings of the talks given at the event.
However, they have collected the slide decks from each of the speakers and assembled them on this page for you
Also, we have some stuff from MeetBSD already:
Youtube Playlist (https://www.youtube.com/playlist?list=PLb87fdKUIo8TAMC2HJLZ7H54edD2BeGWv)
Not all of the sessions are posted yet, but the rest should appear shortly
MeetBSD 2016 Trip Report: Domagoj Stolfa (https://www.freebsdfoundation.org/blog/meetbsd-2016-trip-report-domagoj-stolfa/)
***
Cohabiting FreeBSD and Gentoo Linux on a Common ZFS Volume (https://ericmccorkleblog.wordpress.com/2016/11/15/cohabiting-freebsd-and-gentoo-linux-on-a-common-zfs-volume/)
Eric McCorkle, who has contributed ZFS support to the FreeBSD EFI boot-loader code has posted an in-depth look at how he’s setup dual-boot with FreeBSD and Gentoo on the same ZFS volume.
He starts by giving us some background on how the layout is done. First up, GRUB is used as the boot-loader, allowing boot of both Linux and BSD
The next non-typical thing was using /etc/fstab to manage mount-points, instead of the typical ‘zfs mount’ usage, (apart from /home datasets)
data/home is mounted to /home, with all of its child datasets using the ZFS mountpoint system
data/freebsd and its child datasets house the FreeBSD system, and all have their mountpoints set to legacy
data/gentoo and its child datasets house the Gentoo system, and have their mountpoints set to legacy as well
So, how did he set this up? He helpfully provides an overview of the steps:
Use the FreeBSD installer to create the GPT and ZFS pool
Install and configure FreeBSD, with the native FreeBSD boot loader
Boot into FreeBSD, create the Gentoo Linux datasets, install GRUB
Boot into the Gentoo Linux installer, install Gentoo
Boot into Gentoo, finish any configuration tasks
The rest of the article walks us through the individual commands that make up each of those steps, as well as how to craft a GRUB config file capable of booting both systems.
Personally, since we are using EFI, I would have installed rEFInd, and chain-loaded each systems EFI boot code from there, allowing the use of the BSD loader, but to each their own!
HardenedBSD introduces Safestack into base (https://hardenedbsd.org/article/shawn-webb/2016-11-27/introducing-safestack)
HardenedBSD has integrated SafeStack into its base system and ports tree
SafeStack (http://clang.llvm.org/docs/SafeStack.html) is part of the Code Pointer Integrity (CPI) project within clang.
“SafeStack is an instrumentation pass that protects programs against attacks based on stack buffer overflows, without introducing any measurable performance overhead. It works by separating the program stack into two distinct regions: the safe stack and the unsafe stack. The safe stack stores return addresses, register spills, and local variables that are always accessed in a safe way, while the unsafe stack stores everything else. This separation ensures that buffer overflows on the unsafe stack cannot be used to overwrite anything on the safe stack.”
“As of 28 November 2016, with clang 3.9.0, SafeStack only supports being applied to applications and not shared libraries. Multiple patches have been submitted to clang by third parties to add support for shared libraries.”
SafeStack is only enabled on AMD64
***
pledge(2)… or, how I learned to love web application sandboxing (https://learnbchs.org/pledge.html)
We’ve talked about OpenBSD’s sandboxing mechanism pledge() in the past, but today we have a great article by Kristaps Dzonsons, about how he grew to love it for Web Sandboxing.
+First up, he gives us his opening argument that should make most of you sit up and listen:
I use application-level sandboxing a lot because I make mistakes a lot; and when writing web applications, the price of making mistakes is very dear.
In the early 2000s, that meant using systrace(4) on OpenBSD and NetBSD. Then it was seccomp(2) (followed by libseccomp(3)) on Linux. Then there was capsicum(4) on FreeBSD and sandbox_init(3) on Mac OS X.
All of these systems are invoked differently; and for the most part, whenever it came time to interface with one of them, I longed for sweet release from the nightmare. Please, try reading seccomp(2). To the end. Aligning web application logic and security policy would require an arduous (and usually trial-and-error or worse, copy-and-paste) process. If there was any process at all — if the burden of writing a policy didn't cause me to abandon sandboxing at the start.
And then there was pledge(2).
This document is about pledge(2) and why you should use it and love it. “
+Not convinced yet? Maybe you should take his challenge:
Let's play a drinking game. The challenge is to stay out of the hospital.
1.Navigate to seccomp(2).
2. Read it to the end.
3. Drink every time you don't understand.
For capsicum(4), the challenge is no less difficult. To see these in action, navigate no further than OpenSSH, which interfaces with these sandboxes: sandbox-seccomp-filter.c or sandbox-capsicum.c. (For a history lesson, you can even see sandbox-systrace.c.) Keep in mind that these do little more than restrict resources to open descriptors and the usual necessities of memory, signals, timing, etc. Keep that in mind and be horrified. “
Now Kristaps has his theory on why these are so difficult (NS..), but perhaps there is a better way. He makes the case that pledge() sits right in that sweet-spot, being powerful enough to be useful, but easy enough to implement that developers might actually use it.
All in all, a nice read, check it out! Would love to hear other developer success stories using pledge() as well.
***
News Roundup
Unix history repository, now on GitHub (http://www.osnews.com/story/29513/Unix_history_repository_now_on_GitHub)
OS News has an interesting tidbit on their site today, about the entire commit history of Unix now being available online, starting all the way back in 1970 and bringing us forward to today.
From the README
The history and evolution of the Unix operating system is made available as a revision management repository, covering the period from its inception in 1970 as a 2.5 thousand line kernel and 26 commands, to 2016 as a widely-used 27 million line system. The 1.1GB repository contains about half a million commits and more than two thousand merges. The repository employs Git system for its storage and is hosted on GitHub. It has been created by synthesizing with custom software 24 snapshots of systems developed at Bell Labs, the University of California at Berkeley, and the 386BSD team, two legacy repositories, and the modern repository of the open source FreeBSD system. In total, about one thousand individual contributors are identified, the early ones through primary research. The data set can be used for empirical research in software engineering, information systems, and software archaeology.
This is a fascinating find, especially will be of value to students and historians who wish to look back in time to see how UNIX evolved, and in this repo ultimately turned into modern FreeBSD.
***
Yandex commits improvements to FreeBSD network stack (https://reviews.freebsd.org/D8526)
“Rework ip_tryforward() to use FIB4 KPI.”
This commit brings some code from the experimental routing branch into head
As you can see from the graphs, it offers some sizable improvements in forwarding and firewalled packets per second
commit (https://svnweb.freebsd.org/base?view=revision&revision=309257)
***
The brief history of Unix socket multiplexing – select(2) system call (https://idea.popcount.org/2016-11-01-a-brief-history-of-select2/)
Ever wondered about the details of socket multiplexing, aka the history of select(2)?
Well Marek today gives a treat, with a quick look back at the history that made today’s modern multiplexing possible.
First, his article starts the way all good ones do, presenting the problem in silent-movie form:
In mid-1960's time sharing was still a recent invention. Compared to a previous paradigm - batch-processing - time sharing was truly revolutionary. It greatly reduced the time wasted between writing a program and getting its result. Batch-processing meant hours and hours of waiting often to only see a program error. See this film to better understand the problems of 1960's programmers: "The trials and tribulations of batch processing".
Enter the wild world of the 1970’s, and we’ve now reached the birth of UNIX which tried to solve the batch processing problem with time-sharing.
These days when a program was executed, it could "stall" (block) only on a couple of things1:
+ wait for CPU
+ wait for disk I/O
+ wait for user input (waiting for a shell command) or console (printing data too fast)“
Jump forward another dozen years or so, and the world changes yet again:
This all changed in 1983 with the release of 4.2BSD. This revision introduced an early implementation of a TCP/IP stack and most importantly - the BSD Sockets API.Although today we take the BSD sockets API for granted, it wasn't obvious it was the right API. STREAMS were a competing API design on System V Revision 3.
Coming in along with the sockets API was the select(2) call, which our very own Kirk McKusick gives us some background on:
Select was introduced to allow applications to multiplex their I/O.
Consider a simple application like a remote login. It has descriptors for reading from and writing to the terminal and a descriptor for the (bidirectional) socket. It needs to read from the terminal keyboard and write those characters to the socket. It also needs to read from the socket and write to the terminal. Reading from a descriptor that has nothing queued causes the application to block until data arrives. The application does not know whether to read from the terminal or the socket and if it guesses wrong will incorrectly block. So select was added to let it find out which descriptor had data ready to read. If neither, select blocks until data arrives on one descriptor and then awakens telling which descriptor has data to read.
[...] Non-blocking was added at the same time as select. But using non-blocking when reading descriptors does not work well. Do you go into an infinite loop trying to read each of your input descriptors? If not, do you pause after each pass and if so for how long to remain responsive to input? Select is just far more efficient.
Select also lets you create a single inetd daemon rather than having to have a separate daemon for every service.
The article then wraps up with an interesting conclusion:
> CSP = Communicating sequential processes
In this discussion I was afraid to phrase the core question. Were Unix processes intended to be CSP-style processes? Are file descriptors a CSP-derived "channels"? Is select equivalent to ALT statement?
I think: no. Even if there are design similarities, they are accidental. The file-descriptor abstractions were developed well before the original CSP paper.
It seems that an operating socket API's evolved totally disconnected from the userspace CSP-alike programming paradigms. It's a pity though. It would be interesting to see an operating system coherent with the programming paradigms of the user land programs.
A long (but good) read, and worth your time if you are interested in the history how modern multiplexing came to be.
***
How to start CLion on FreeBSD? (https://intellij-support.jetbrains.com/hc/en-us/articles/206525024-How-to-start-CLion-on-FreeBSD)
CLion (pronounced "sea lion") is a cross-platform C and C++ IDE
By default, the Linux version comes bundled with some binaries, which obviously won’t work with the native FreeBSD build
Rather than using Linux emulation, you can replace these components with native versions
pkg install openjdk8 cmake gdb
Edit clion-2016.3/bin/idea.properties and change run.processes.with.pty=false
Start CLion and open Settings | Build, Execution, Deployment | Toolchains
Specify CMake path: /usr/local/bin/cmake and GDB path: /usr/local/bin/gdb
Without a replacement for fsnotifier, you will get a warning that the IDE may be slow to detect changes to files on disk
But, someone has already written a version of fsnotifier that works on FreeBSD and OpenBSD
fsnotifier for OpenBSD and FreeBSD (https://github.com/idea4bsd/fsnotifier) -- The fsnotifier is used by IntelliJ for detecting file changes. This version supports FreeBSD and OpenBSD via libinotify and is a replacement for the bundled Linux-only version coming with the IntelliJ IDEA Community Edition.
***
Beastie Bits
TrueOS Pico – FreeBSD ARM/RPi Thin Clients (https://www.trueos.org/trueos-pico/)
A Puppet package provider for FreeBSD's PkgNG package manager. (https://github.com/xaque208/puppet-pkgng)
Notes from November London *BSD meetup (http://mailman.uk.freebsd.org/pipermail/ukfreebsd/2016-November/014059.html)
SemiBug meeting on Dec 20th (http://lists.nycbug.org/pipermail/semibug/2016-November/000131.html)
Feedback/Questions
Erno - SSH without password (http://pastebin.com/SMvxur9v)
Jonathan - Magical ZFS (http://pastebin.com/5ETL7nmj)
George - TrueOS (http://pastebin.com/tSVvaV9e)
Mohammad - Jails IP (http://pastebin.com/T8nUexd1)
Gibheer - BEs (http://pastebin.com/YssXXp70)
***
| |||
| 18 Nov 2021 | 429: Advanced ZFS Snapshots | 00:39:04 | |
FreeBSD Foundation October Fundraising Update, Advanced ZFS Snapshots, Full WireGuard setup with OpenBSD, MidnightBSD a Linux Alternative, FreeBSD Audio, Tuning Power Consumption on FreeBSD Laptops, Thoughts on Spelling Fixes, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
FreeBSD Foundation October 2021 Fundraising Update (https://freebsdfoundation.org/blog/freebsd-foundation-october-2021-fundraising-update/)
Advanced ZFS Snapshots (https://klarasystems.com/articles/advanced-zfs-snapshots/)
News Roundup
Full WireGuard setup with OpenBSD (https://dataswamp.org/~solene/2021-10-09-openbsd-wireguard-exit.html)
MidnightBSD a Linux Alternative (https://www.makeuseof.com/midnightbsd-linux-desktop-alternative/)
FreeBSD Audio (https://meka.rs/blog/2021/10/12/freebsd-audio/)
Tuning Power Consumption on FreeBSD Laptops and Intel Speed Shift (6th Gen and Later) (https://www.neelc.org/posts/freebsd-speed-shift-laptop/)
Some Thoughts on Spelling Fixes (http://bsdimp.blogspot.com/2021/10/spelling-fixes-some-advice.html)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Bens feedback to Benedict's feedback to Bens question about zpoolboy (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/429/feedback/Bens%20feedback%20to%20Benedicts%20feedback%20to%20Bens%20question%20about%20zpoolboy.md)
hcddbz - Old Technical Books (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/429/feedback/hcddbz%20-%20Old%20Technical%20Books.md)
jason - a jails question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/429/feedback/jason%20-%20a%20jails%20question.md)
***
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 03 Nov 2022 | 479: OpenBSD Docker Host | 00:42:03 | |
EuroBSDcon 2022 as first BSD conference, Red Hat’s OpenShift vs FreeBSD Jails, Running a Docker Host under OpenBSD using vmd(8), history of sending signals to Unix process groups, Toolchains adventures - Q3 2022, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
EuroBSDCon 2022, my first BSD conference (and how they are different) (https://eerielinux.wordpress.com/2022/09/25/eurobsdcon-2022-my-first-bsd-conference-and-how-they-are-different/)
Red Hat’s OpenShift vs FreeBSD Jails (https://klarasystems.com/articles/red-hats-openshift-vs-freebsd-jails/)
News Roundup
The history of sending signals to Unix process groups (https://utcc.utoronto.ca/~cks/space/blog/unix/ProcessGroupsAndSignals)
Running a Docker Host under OpenBSD using vmd(8) (https://www.tumfatig.net/2022/running-docker-host-openbsd-vmd/)
Toolchains adventures - Q3 2022 (https://www.cambus.net/toolchains-adventures-q3-2022/)
Beastie Bits
-current has moved to 7.2 (https://undeadly.org/cgi?action=article;sid=20220912055003)
Several /sbin daemons are now dynamically-linked (http://undeadly.org/cgi?action=article;sid=20220830052924)
Announcing the pkgsrc 2022Q3 branch (https://mail-index.netbsd.org/netbsd-announce/2022/09/29/msg000341.html)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Hans - datacenters and dust (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/476/feedback/Hans%20-%20datacenters%20and%20dust.md)
Tim - Boot issue (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/476/feedback/Tim%20-%20Boot%20issue.md)
aaron- dwm tiling (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/476/feedback/aaron-%20dwm%20tiling%20.md)
***
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 30 Apr 2014 | 35: Puffy Firewall | 01:19:23 | |
We're back again! On this week's packed show, we've got one of the biggest tutorials we've done in a while. It's an in-depth look at PF, OpenBSD's firewall, with some practical examples and different use cases. We'll also be talking to Peter Hansteen about the new edition of "The Book of PF." Of course, we've got news and answers to your emails too, on BSD Now - the place to B.. SD.
This episode was brought to you by
Headlines
ALTQ removed from PF (http://undeadly.org/cgi?action=article&sid=20140419151959)
Kicking off our big PF episode...
The classic packet queueing system, ALTQ, was recently removed from OpenBSD -current
There will be a transitional phase between 5.5 and 5.6 where you can still use it by replacing the "queue" keyword with "oldqueue" in your pf.conf
As of 5.6, due about six months from now, you'll have to change your ruleset to the new syntax if you're using it for bandwidth shaping
After more than ten years, bandwidth queueing has matured quite a bit and we can finally put ALTQ to rest, in favor of the new queueing subsystem
This doesn't affect FreeBSD, PCBSD, NetBSD or DragonflyBSD since all of their PFs are older and maintained separately.
***
FreeBSD Quarterly Status Report (https://www.freebsd.org/news/status/report-2014-01-2014-03.html)
The quarterly status report from FreeBSD is out, detailing some of the project's ongoing tasks
Some highlights include the first "stable" branch of ports, ARM improvements (including SMP), bhyve improvements, more work on the test suite, desktop improvements including the new vt console driver and UEFI booting support finally being added
We've got some specific updates from the cluster admin team, core team, documentation team, portmgr team, email team and release engineering team
LOTS of details and LOTS of topics to cover, give it a read
***
OpenBSD's OpenSSL rewrite continues with m2k14 (http://undeadly.org/cgi?action=article&sid=20140417184158)
A mini OpenBSD hackathon (http://www.openbsd.org/hackathons.html) begins in Morocco, Africa
You can follow the changes in the -current CVS log (http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libssl/src/ssl/), but a lot of work (http://undeadly.org/cgi?action=article&sid=20140418063443) is mainly going towards the OpenSSL cleaning
We've got two trip (http://undeadly.org/cgi?action=article&sid=20140429121423) reports (http://undeadly.org/cgi?action=article&sid=20140425115340) so far, hopefully we'll have some more to show you in a future episode
You can see some of the more interesting quotes (http://opensslrampage.org/) from the tear-down or see everything (http://freshbsd.org/commit/openbsd/e5136d69ece4682e6167c8f4a8122270236898bf)
Apparently (http://undeadly.org/cgi?action=article&sid=20140423045847) they are going to call the fork "LibreSSL (https://news.ycombinator.com/item?id=7623789)" ....
What were the OpenSSL developers thinking (http://freshbsd.org/commit/openbsd/e5136d69ece4682e6167c8f4a8122270236898bf)? The RSA private key was used to seed the entropy!
We also got some mainstream news coverage (http://www.zdnet.com/openbsd-forks-prunes-fixes-openssl-7000028613/) and another post from Ted (http://www.tedunangst.com/flak/post/origins-of-libressl) about the history of the fork
Definitely consider donating to the OpenBSD foundation (http://www.openbsdfoundation.org/donations.html), this fork will benefit all the other BSDs too
***
NetBSD 6.1.4 and 6.0.5 released (https://blog.netbsd.org/tnf/entry/netbsd_6_1_4_and)
New updates for the 6.1 and 6.0 branches of NetBSD, focusing on bugfixes
The main update is - of course - the heartbleed vulnerability
Also includes fixes for other security issues and even a kernel panic... on Atari
Patch your Ataris right now, this is serious business
***
Interview - Peter Hansteen - peter@bsdly.net (mailto:peter@bsdly.net) / @pitrh (https://twitter.com/pitrh)
The Book of PF: 3rd edition
Tutorial
BSD Firewalls: PF (http://www.bsdnow.tv/tutorials/pf)
News Roundup
New Xorg now the default in FreeBSD (https://svnweb.freebsd.org/ports?view=revision&revision=351411)
For quite a while now, FreeBSD has had two versions of X11 in ports
The older, stable version was the default, but you could install a newer one by having "WITHNEWXORG" in /etc/make.conf
They've finally made the switch for 10-STABLE and 9-STABLE
Check this wiki page (https://wiki.freebsd.org/Graphics) for more info
***
GSoC-accepted BSD projects (https://www.google-melange.com/gsoc/org2/google/gsoc2014/openbsdfoundation)
The Google Summer of Code team has got the list of accepted project proposals uploaded so we can see what's planned
OpenBSD's list includes DHCP configuration parsing improvements, systemd replacements, porting capsicum, GPT and UEFI support, and modernizing the DHCP daemon
The FreeBSD list (https://www.google-melange.com/gsoc/org2/google/gsoc2014/freebsd) was also posted
Theirs includes porting FreeBSD to the Android emulator, CTF in the kernel debugger, improved unicode support, converting firewall rules to a C module, pkgng improvements, MicroBlaze support, PXE fixes, bhyve caching, bootsplash and lots more
Good luck to all the students participating, hopefully they become full time BSD users
***
Complexity of FreeBSD VFS using ZFS as an example (http://www.hybridcluster.com/blog/complexity-freebsd-vfs-using-zfs-example-part-2/)
HybridCluster posted the second part of their VFS and ZFS series
This new post has lots of technical details once again, definitely worth reading if you're a ZFS guy
Of course, also watch episode 24 (http://www.bsdnow.tv/episodes/2014_02_12-the_cluster_the_cloud) for our interview with HybridCluster - they do really interesting stuff
***
PCBSD weekly digest (http://blog.pcbsd.org/2014/04/weekly-feature-digest-26-the-lumina-project-and-preload/)
Preload has been ported over, it's a daemon that prefetches applications
PCBSD is developing their own desktop environment, Lumina (there's also an FAQ (http://blog.pcbsd.org/2014/04/quick-lumina-desktop-faq/))
It's still in active development, but you can try it out by installing from ports
We'll be showing a live demo of it in a few weeks (when development settles down a bit)
Some kid in Australia subjects his poor mother to being on camera (https://www.youtube.com/watch?v=ETxhbf3-z18) while she tries out PCBSD and gives her impressions of it
***
| |||
| 04 Oct 2017 | 214: The history of man, kind | 01:30:20 | |
The costs of open sourcing a project are explored, we discover why PS4 downloads are so slow, delve into the history of UNIX man pages, and more.
This episode was brought to you by
Headlines
The Cost Of Open Sourcing Your Project (https://meshedinsights.com/2016/09/20/open-source-unlikely-to-be-abandonware/)
Accusing a company of “dumping” their project as open source is probably misplaced – it’s an expensive business no-one would do frivolously.
If you see an active move to change software licensing or governance, it’s likely someone is paying for it and thus could justify the expense to an executive.
A Little History
Some case study cameos may help. From 2004 onwards, Sun Microsystems had a policy of all its software moving to open source. The company migrated almost all products to open source licenses, and had varying degrees of success engaging communities around the various projects, largely related to the outlooks of the product management and Sun developers for the project.
Sun occasionally received requests to make older, retired products open source. For example, Sun acquired a company called Lighthouse Design which created a respected suite of office productivity software for Steve Jobs’ NeXT platform. Strategy changes meant that software headed for the vault (while Jonathan Schwartz, a founder of Lighthouse, headed for the executive suite). Members of the public asked if Sun would open source some of this software, but these requests were declined because there was no business unit willing to fund the move.
When Sun was later bought by Oracle, a number of those projects that had been made open source were abandoned. “Abandoning” software doesn’t mean leaving it for others; it means simply walking away from wherever you left it. In the case of Sun’s popular identity middleware products, that meant Oracle let the staff go and tried to migrate customers to other products, while remaining silent in public on the future of the project. But the code was already open source, so the user community was able to pick up the pieces and carry on, with help from Forgerock.
It costs a lot of money to open source a mature piece of commercial software, even if all you are doing is “throwing a tarball over the wall”. That’s why companies abandoning software they no longer care about so rarely make it open source, and those abandoning open source projects rarely move them to new homes that benefit others.
If all you have thought about is the eventual outcome, you may be surprised how expensive it is to get there. Costs include: For throwing a tarball over the wall:
Legal clearance.
Having the right to use the software is not the same as giving everyone in the world an unrestricted right to use it and create derivatives. Checking every line of code to make sure you have the rights necessary to release under an OSI-approved license is a big task requiring high-value employees on the “liberation team”. That includes both developers and lawyers; neither come cheap.
Repackaging.
To pass it to others, a self-contained package containing all necessary source code, build scripts and non-public source and tool dependencies has to be created since it is quite unlikely to exist internally. Again, the liberation team will need your best developers.
Preserving provenance.
Just because you have confidence that you have the rights to the code, that doesn’t mean anyone else will. The version control system probably contains much of the information that gives confidence about who wrote which code, so the repackaging needs to also include a way to migrate the commit information.
Code cleaning.
The file headers will hopefully include origin information but the liberation team had better check. They also need to check the comments for libel and profanities, not to mention trade secrets (especially those from third parties) and other IP issues.
For a sustainable project, all the above plus:
Compliance with host governance.
It is a fantastic idea to move your project to a host like Apache, Conservancy, Public Software and so on. But doing so requires preparatory work. As a minimum you will need to negotiate with the new host organisation, and they may well need you to satisfy their process requirements. Paperwork obviously, but also the code may need conforming copyright statements and more. That’s more work for your liberation team.
Migration of rights.
Your code has an existing community who will need to migrate to your new host. That includes your staff – they are community too! They will need commit rights, governance rights, social media rights and more. Your liberation team will need your community manager, obviously, but may also need HR input.
Endowment.
Keeping your project alive will take money. It’s all been coming from you up to this point, but if you simply walk away before the financial burden has been accepted by the new community and hosts there may be a problem. You should consider making an endowment to your new host to pay for their migration costs plus the cost of hosting the community for at least a year.
Marketing.
Explaining the move you are making, the reasons why you are making it and the benefits for you and the community is important. If you don’t do it, there are plenty of trolls around who will do it for you. Creating a news blog post and an FAQ — the minimum effort necessary — really does take someone experienced and you’ll want to add such a person to your liberation team.
Motivations
There has to be some commercial reason that makes the time, effort and thus expense worth incurring. Some examples of motivations include:
Market Strategy.
An increasing number of companies are choosing to create substantial, openly-governed open source communities around software that contributes to their business. An open multi-stakeholder co-developer community is an excellent vehicle for innovation at the lowest cost to all involved. As long as your market strategy doesn’t require creating artificial scarcity.
Contract with a third party.
While the owner of the code may no longer be interested, there may be one or more parties to which they owe a contractual responsibility. Rather than breaching that contract, or buying it out, a move to open source may be better. Some sources suggest a contractual obligation to IBM was the reason Oracle abandoned OpenOffice.org by moving it over to the Apache Software Foundation for example.
Larger dependent ecosystem.
You may have no further use for the code itself, but you may well have other parts of your business which depend on it. If they are willing to collectively fund development you might consider an “inner source” strategy which will save you many of the costs above. But the best way to proceed may well be to open the code so your teams and those in other companies can fund the code.
Internal politics.
From the outside, corporations look monolithic, but from the inside it becomes clear they are a microcosm of the market in which they exist. As a result, they have political machinations that may be addressed by open source. One of Oracle’s motivations for moving NetBeans to Apache seems to have been political. Despite multiple internal groups needing it to exist, the code was not generating enough direct revenue to satisfy successive executive owners, who allegedly tried to abandon it on more than one occasion. Donating it to Apache meant that couldn’t happen again.
None of this is to say a move to open source guarantees the success of a project. A “Field of Dreams” strategy only works in the movies, after all. But while it may be tempting to look at a failed corporate liberation and describe it as “abandonware”, chances are it was intended as nothing of the kind.
Why PS4 downloads are so slow (https://www.snellman.net/blog/archive/2017-08-19-slow-ps4-downloads/)
From the blog that brought us “The origins of XXX as FIXME (https://www.snellman.net/blog/archive/2017-04-17-xxx-fixme/)” and “The mystery of the hanging S3 downloads (https://www.snellman.net/blog/archive/2017-07-20-s3-mystery/)”, this week it is: “Why are PS4 downloads so slow?”
Game downloads on PS4 have a reputation of being very slow, with many people reporting downloads being an order of magnitude faster on Steam or Xbox. This had long been on my list of things to look into, but at a pretty low priority. After all, the PS4 operating system is based on a reasonably modern FreeBSD (9.0), so there should not be any crippling issues in the TCP stack.
The implication is that the problem is something boring, like an inadequately dimensioned CDN. But then I heard that people were successfully using local HTTP proxies as a workaround. It should be pretty rare for that to actually help with download speeds, which made this sound like a much more interesting problem.
Before running any experiments, it's good to have a mental model of how the thing we're testing works, and where the problems might be. If nothing else, it will guide the initial experiment design.
The speed of a steady-state TCP connection is basically defined by three numbers. The amount of data the client is will to receive on a single round-trip (TCP receive window), the amount of data the server is willing to send on a single round-trip (TCP congestion window), and the round trip latency between the client and the server (RTT). To a first approximation, the connection speed will be:
speed = min(rwin, cwin) / RTT
With this model, how could a proxy speed up the connection?
The speed through the proxy should be the minimum of the speed between the client and proxy, and the proxy and server. It should only possibly be slower
With a local proxy the client-proxy RTT will be very low; that connection is almost guaranteed to be the faster one. The improvement will have to be from the server-proxy connection being somehow better than the direct client-server one. The RTT will not change, so there are just two options: either the client has a much smaller receive window than the proxy, or the client is somehow causing the server's congestion window to decrease. (E.g. the client is randomly dropping received packets, while the proxy isn't).
After setting up a test rig, where the PS4’s connection was bridged through a linux box so packets could be captured, and artificial latency could be added, some interested results came up:
The differences in receive windows at different times are striking. And more important, the changes in the receive windows correspond very well to specific things I did on the PS4
When the download was started, the game Styx: Shards of Darkness was running in the background (just idling in the title screen). The download was limited by a receive window of under 7kB. This is an incredibly low value; it's basically going to cause the downloads to take 100 times longer than they should. And this was not a coincidence, whenever that game was running, the receive window would be that low.
Having an app running (e.g. Netflix, Spotify) limited the receive window to 128kB, for about a 5x reduction in potential download speed.
Moving apps, games, or the download window to the foreground or background didn't have any effect on the receive window.
Playing an online match in a networked game (Dreadnought) caused the receive window to be artificially limited to 7kB.
I ran a speedtest at a time when downloads were limited to 7kB receive window. It got a decent receive window of over 400kB; the conclusion is that the artificial receive window limit appears to only apply to PSN downloads.
When a game was started (causing the previously running game to be stopped automatically), the receive window could increase to 650kB for a very brief period of time. Basically it appears that the receive window gets unclamped when the old game stops, and then clamped again a few seconds later when the new game actually starts up.
I did a few more test runs, and all of them seemed to support the above findings. The only additional information from that testing is that the rest mode behavior was dependent on the PS4 settings. Originally I had it set up to suspend apps when in rest mode. If that setting was disabled, the apps would be closed when entering in rest mode, and the downloads would proceed at full speed.
The PS4 doesn't make it very obvious exactly what programs are running. For games, the interaction model is that opening a new game closes the previously running one. This is not how other apps work; they remain in the background indefinitely until you explicitly close them.
So, FreeBSD and its network stack are not to blame
Sony used a poor method to try to keep downloads from interfering with your gameplay
The impact of changing the receive window is highly dependant upon RTT, so it doesn’t work as evenly as actual traffic shaping or queueing would.
An interesting deep dive, it is well worth reading the full article and checking out the graphs
***
OpenSSH 7.6 Released (http://www.openssh.com/releasenotes.html#7.6)
From the release notes:
This release includes a number of changes that may affect existing
configurations:
ssh(1): delete SSH protocol version 1 support, associated
configuration options and documentation.
ssh(1)/sshd(8): remove support for the hmac-ripemd160 MAC.
ssh(1)/sshd(8): remove support for the arcfour, blowfish and CAST
Refuse RSA keys <1024 bits in length and improve reporting for keys that do not meet
this requirement.
ssh(1): do not offer CBC ciphers by default.
Changes since OpenSSH 7.5
This is primarily a bugfix release. It also contains substantial internal refactoring.
Security: sftp-server(8): in read-only mode, sftp-server was incorrectly permitting creation of zero-length files. Reported by Michal Zalewski.
New features:
ssh(1): add RemoteCommand option to specify a command in the ssh config file instead of giving it on the client's command line. This allows the configuration file to specify the command that will be executed on the remote host.
sshd(8): add ExposeAuthInfo option that enables writing details of the authentication methods used (including public keys where applicable) to a file that is exposed via a $SSHUSERAUTH environment variable in the subsequent session.
ssh(1): add support for reverse dynamic forwarding. In this mode, ssh will act as a SOCKS4/5 proxy and forward connections to destinations requested by the remote SOCKS client. This mode is requested using extended syntax for the -R and RemoteForward options and, because it is implemented solely at the client, does not require the server be updated to be supported.
sshd(8): allow LogLevel directive in sshd_config Match blocks;
ssh-keygen(1): allow inclusion of arbitrary string or flag certificate extensions and critical options.
ssh-keygen(1): allow ssh-keygen to use a key held in ssh-agent as a CA when signing certificates.
ssh(1)/sshd(8): allow IPQoS=none in ssh/sshd to not set an explicit ToS/DSCP value and just use the operating system default.
ssh-add(1): added -q option to make ssh-add quiet on success.
ssh(1): expand the StrictHostKeyChecking option with two new settings. The first "accept-new" will automatically accept hitherto-unseen keys but will refuse connections for changed or invalid hostkeys. This is a safer subset of the current behaviour of StrictHostKeyChecking=no. The second setting "off", is a synonym for the current behaviour of StrictHostKeyChecking=no: accept new host keys, and continue connection for hosts with incorrect hostkeys. A future release will change the meaning of StrictHostKeyChecking=no to the behaviour of "accept-new".
ssh(1): add SyslogFacility option to ssh(1) matching the equivalent option in sshd(8).
Check out the bugfixes and portability sections, too.
***
News Roundup
FreeBSD comes to FiFo 0.9.3 with vmadm (https://blog.project-fifo.net/freebsd-in-fifo-0-9-3/)
What is Project FiFo? It’s an Open Source SmartOS Cloud Management and orchestration software. FiFo can be installed on SmartOS zones, running on standard compute nodes. There is no need for dedicated hardware or server roles.
FiFo 0.9.3 has been in the works for a while, and it comes with quite a few new features. With our last release, we started experimenting with FreeBSD support. Since then much work has gone into improving this. We also did something rather exciting with the mystery box! However, more on that in a later post.
The stable release of 0.9.3 will land within a few days with only packaging and documentation tasks left to do. Part of this means that we’ll have packages for all major components that work natively on BSD. There is no more need for a SmartOS box to run the components!
When we introduced FreeBSD support last version we marked it as an experimental feature. We needed to try out and experiment what works and what does not. Understand the way FreeBSD does things, what tools exist, and how those align with our workflow. Bottomline we were not even sure BSD support was a thing in the future.
We are happy to announce that with 0.9.3 we are now sure BSD support is a thing, and it is here to remain. That said it was good that we experimented in the last release, we did some significant changes to what we have now. When first looking at FreeBSD we went ahead and used existing tooling, namely iocage, to manage jails. It turns out the tooling around jails is not on par with what exists on illumos and especially SmartOS. The goodness of vmadm as a CLI for managing zones is just unparalleled. So we do what every (in)sane person would do!
So with 0.9.3, we did what every (in)sane person would do! We implemented a version of vmadm that would work with FreeBSD and jails and keep the same CLI. Our clone works completely stand alone; vmadm is a compiled binary, written in rust which is blazing fast! The design takes up lessons learned from both zoneadm and vmadm in illumos/SmartOS for how things work instead of trying to reinvent the wheel. Moreover, while we love giving the FreeBSD community a tool we learned to love on SmartOS this also makes things a lot easier for us. FiFo now can use the same logic on SmartOS and FreeBSD as the differences are abstracted away inside of vmadm. That said there are a few notable differences.
First of all, vmadm uses datasets the same way it does on SmartOS. However, there is no separate imgadm tool. Instead, we encapsulate the commands under vmadm images. To make this work we also provide a dataset server with base images for FreeBSD that used the same API as SmartOS dataset servers. Second, we needed to work around some limitations in VNET to make jails capable of being fully utilized in multi-tenancy environments.
Nested vnet jails on freebsd While on illumos a virtual nic can be bound to an IP that can not be changed from inside the zone, VNET does not support this. Preventing tenants from messing with IP settings is crucial from a security standpoint!
To work around that each jail created by vmadm are two jails: a minimal outer jail with nothing but a VNET interface, no IP or anything and an internal one that runs the user code. This outer jail then creates an inner jail with an inherited NIC that gets a fixed IP, combining both the security of a VNET jail as well as the security of a fixed IP interface.
The nested jail layout resembles the way that SmartOS handles KVM machines, running KVM inside a zone. So in addition to working around VNET limitations, this already paves the way for bhyve nested in jails that might come in a future release. We hope to leverage the same two-step with just a different executable started in the outer jail instead of the jail command itself.
History of UNIX Manpages (https://manpages.bsd.lv/history.html)
Where do UNIX manpages come from? Who introduced the section-based layout of NAME, SYNOPSIS, and so on? And for manpage source writers and readers: where were those economical two- and three-letter instructions developed? The many accounts available on the Internet lack citations and are at times inconsistent.
In this article, I reconstruct the history of the UNIX manpage based on source code, manuals, and first-hand accounts.
Special thanks to Paul Pierce for his CTSS source archive; Bernard Nivelet for the Multics Internet Server; the UNIX Heritage Society for their research UNIX source reconstruction; Gunnar Ritter for the Heirloom Project sources; Alcatel-Lucent Bell Labs for the Plan 9 sources; BitSavers for their historical archive; and last but not least, Rudd Canaday, James Clarke, Brian Kernighan, Douglas McIlroy, Nils-Peter Nelson, Jerome Saltzer, Henry Spencer, Ken Thompson, and Tom Van Vleck for their valuable contributions.
Please see the Copyright section if you plan on reproducing parts of this work.
People:
Abell, Vic
Canaday, Rudd
Capps, Dennis
Clarke, James
Dzonsons, Kristaps
Kernighan, Brian
Madnick, Stuart
McIlroy, Douglas
Morris, Robert
Ossanna, Joseph F.
Ritchie, Dennis
Ritter, Gunnar
Saltzer, Jerome H.
Spencer, Henry
Thompson, Ken
***
BSDCam 2017 Trip Report: Mathieu Arnold (https://www.freebsdfoundation.org/blog/bsdcam-2017-trip-report-mathieu-arnold/)
It seems that every time I try to go to England using the Eurostar, it gets delayed between 30 minutes and 2 hours. This year, it got my 45 minute layover down to 10 minutes. Luckily, King’s Cross is literally across the street from Saint Pancras, and I managed to get into my second train just in time.
I arrived in Cambridge on Tuesday right on time for tea. A quick walk from the station got me to St Catharine’s College. This year, we were in a different building for the rooms, so I listened to the convoluted explanation the porter gave me to get to my room, I managed to get there without getting lost more than once. That evening was almost organized as we got together for dinner at the usual pub, the Maypole.
Wednesday:
The weather is lovely, and it is a good thing as there is a 25-30 minute walk from the College to the Computer Laboratory where the devsummit happens. The first morning is for deciding what we are going to talk about for the rest of the week, so we all go in turn introducing ourselves and voicing about what we would like to talk about. There are a few subjects that are of interest to me, so I listen to the toolchain discussions while writing new bits for the Porter’s Handbook.
Thursday:
I spent most of the day writing documentation, and talked a bit with a couple of DocEng members about joining the team as I would like to give some love to the build framework that has not been touched in a long time. At the end of the afternoon is a packaging session, we talked about the status of package in base, which is not really going anywhere right now. On the ports side, three aspects that are making good progress include, package flavors, sub packages, and migrating some base libraries to private libraries, which is a nightmare because of openssl, and kerberos, and pam. That evening, we had the formal diner at St John’s College, I love those old buildings that reminds me of Hogwarts. (I am sure there is a quidditch pitch somewhere nearby.)
Friday:
Last day. I continued to write documentation, while listening to a provisioning session. It would be great to have bhyve support in existing orchestration tools like vagrant, openstack, or maybe ganeti. We end the day, and the devsummit with short talks, some very interesting, some going way over my head.
The weekend is here. I spent most of Saturday strolling in some of the numerous Cambridge parks, gardens, greens, fens… and I worked on a knitting pattern in the evening. On Sunday, I ate my last full gargantuan english breakfast of the year, and then back in a train to King’s Cross, and a Eurostar (this one on time) back to Paris.
I would like to thank the FreeBSD Foundation for making this trip possible for me.
GSoC 2017 Reports: Add SUBPACKAGES support to pkgsrc, part 1 (https://blog.netbsd.org/tnf/entry/gsoc_2017_reports_add_code)
Introduction
SUBPACKAGES (on some package systems they are known as multi-packages, but this term for pkgsrc is already used by packages that can be built against several versions (e.g. Python, PHP, Ruby packages)) consist in generating multiple binary packages from a single pkgsrc package. For example, from a pkgsrc package - local/frobnitzem - we will see how to generate three separate binary packages: frobnitzem-foo, frobnitzem-bar and frobnitzem-baz.
This can be useful to separate several components of binary packages (and avoid to run the extract and configure phase two times!), for debugpkgs (so that all *.debug files containing debug symbols are contained in a separate -debugpkg package that can be installed only when it is needed), etc..
An high-level look at how SUBPACKAGES support is implemented
Most of the changes needed are in mk/pkgformat/pkg/ hierarchy (previously known as mk/flavour and then renamed and generalized to other package formats during Anton Panev's Google Summer of Code 2011).
The code in mk/pkgformat/${PKGFORMAT}/ handle the interaction of pkgsrc with the particular ${PKGFORMAT}, e.g. for pkg populate meta-data files used by pkgcreate(1), install/delete packages via pkgadd(1), and pkg_delete(1), etc.
Conclusion
In this first part of this blog post series we have seen what are SUBPACKAGES, when and why they can be useful.
We have then seen a practical example of them taking a very trivial package and learned how to "subpackage-ify" it.
Then we have described - from an high-level perspective - the changes needed to the pkgsrc infrastructure for the SUBPACKAGES features that we have used. If you are more interested in them please give a look to the pkgsrc debugpkg branch that contains all work done described in this blog post.
In the next part we will see how to handle DEPENDS and buildlink3 inclusion for subpackages.
**
Beastie Bits
First partial boot of FreeBSD on Power8 (http://dmesgd.nycbug.org/index.cgi?do=view&id=3329)
The new TNF Board of Directors are installed and patched for 2017. (https://blog.netbsd.org/tnf/entry/the_new_tnf_board_of)
Open Source Summit 2017 October 23-26, 2017, Prague, Czech Republic Giovanni Bechis will give a talk about seccomp(2) vs pledge(2) (https://osseu17.sched.com/event/BxJw/seccomp2-vs-pledge2-giovanni-bechis-snb-srl)
My first patch to OpenBSD (http://nanxiao.me/en/my-first-patch-to-openbsd/)
Feedback/Questions
Brian - OPNSense Facebook Group (http://dpaste.com/35WA42Z#wrap)
Mark Felder - ZFS Health via SNMP (http://dpaste.com/0B8QH2W)
Matt - Cantrill Presentation (http://dpaste.com/1D9WTHV#wrap)
***
| |||
| 07 Feb 2018 | 232: FOSDEM 2018 | 01:35:06 | |
We talk about our recent trip to FOSDEM, we discuss the pros and cons of permissive licensing, cover the installation of OpenBSD on a dedibox with full-disk encryption, the new Lumina guide repository, and we explain ZFS vs. OpenZFS.
This episode was brought to you by
Headlines
[FOSDEM Trip report]
Your BSDNow hosts were both at FOSDEM in Brussels, Belgium over the weekend.
On the friday before FOSDEM, we held a FreeBSD devsummit (3rd consecutive year), sponsored by the FreeBSD Foundation and organized by Benedict (with the help from Kristof Provost, who did it in previous years but could not make it this year). We had 21 people attend, a good mixture of FreeBSD committers (mostly ports) and guests. After introductions, we collected topics and discussed various topics, including a new plan for a future FreeBSD release roadmap (more frequent releases, so that features from HEAD can be tried out earlier in RELEASES). The devsummit concluded with a nice dinner in a nearby restaurant.
On Saturday, first day of FOSDEM, we set up the FreeBSD Foundation table with flyers, stickers, FreeBSD Journal print editions, and a small RPI 3 demo system that Deb Goodkin brought. Our table was located next to the Illumos table like last year. This allowed us to continue the good relationship that we have with the Illumos people and Allan helped a little bit getting bhyve to run on Illumos with UEFI. Meanwhile, our table was visited by a lot of people who would ask questions about FreeBSD, take info material, or talk about their use cases. We were busy refilling the table throughout the day and luckily, we had many helpers at the table. Some items we had ran out in the early afternoon, an indicator of how popular they were.
Saturday also featured a BSD devroom (https://twitter.com/fosdembsd), organized by Rodrigo Osorio. You can find the list of talks and the recordings on the BSD Devroom schedule (https://fosdem.org/2018/schedule/track/bsd/). The room was very crowded and popular. Deb Goodkin gave the opening talk with an overview of what the Foundation is doing to change the world. Other speakers from various BSD projects presented their talks after that with a range of topics. Among them, Allan gave his talk about ZFS: Advanced Integration (https://fosdem.org/2018/schedule/event/zfs_advanced_integration/), while Benedict presented his Reflections on Teaching a Unix Class With FreeBSD (https://fosdem.org/2018/schedule/event/reflections_on_reaching_unix_class_with_freebsd/).
Sunday was just as busy on the FreeBSD table as Saturday and we finally ran out of stickers and some other goodies. We were happy with the results of the two days. Some very interesting conversations at the table about FreeBSD took place, some of which we’re going to follow up afterwards.
Check out the FOSDEM schedule as many talk recordings are already available, and especially the ones from the BSD devroom if you could not attend the conference. We would like to thank everyone who attended the FreeBSD devsummit, who helped out at the FreeBSD table and organized the BSD devroom. Also, thanks to all the speakers, organizers, and helping hands making FOSDEM another success this year.
***
NetBSD kernel wscons IOCTL vulnerable bug class (http://blog.infosectcbr.com.au/2018/01/netbsd-kernel-wscons-ioctl-vulnerable.html)
I discovered this bug class during the InfoSect public code review session we ran looking specifically at the NetBSD kernel. I found a couple of these bugs and then after the session was complete, I went back and realised the same bug was scattered in other drivers. In total, 17 instances of this vulnerability and its variants were discovered.
In all fairness, I came across this bug class during my kernel audits in 2002 and most instances were patched. It just seems there are more bugs now in NetBSD while OpenBSD and FreeBSD have practically eliminated them.
See slide 41 in http://www.blackhat.com/presentations/bh-usa-03/bh-us-03-cesare.pdf (http://www.blackhat.com/presentations/bh-usa-03/bh-us-03-cesare.pdf) for exactly the same bug (class) 16 years ago.
The format of the this blog post is as follows:
Introduction
Example of the Bug Class
How to Fix
How to Detect Automatically with Coccinelle
More Bugs
Conclusion
These source files had bugs
./dev/tc/tfb.c
./dev/ic/bt485.c
./dev/pci/radeonfb.c
./dev/ic/sti.c
./dev/sbus/tcx.c
./dev/tc/mfb.c
./dev/tc/sfb.c
./dev/tc/stic.c
./dev/tc/cfb.c
./dev/tc/xcfb.c
./dev/tc/sfbplus.c
./arch/arm/allwinner/awin_debe.c
./arch/arm/iomd/vidcvideo.c
./arch/pmax/ibus/pm.c
./dev/ic/igfsb.c
./dev/ic/bt463.c
./arch/luna68k/dev/lunafb.c
Reporting of the bugs was easy. In less than a week from reporting the specific instances of each bug, patches were committed into the mainline kernel. Thanks to Luke Mewburn from NetBSD for coming to the code review session at InfoSect and coordinating with the NetBSD security team.
The patches to fix these issues are in NetBSD: https://mail-index.netbsd.org/source-changes/2018/01/24/msg091428.html (https://mail-index.netbsd.org/source-changes/2018/01/24/msg091428.html)
"Permissive licensing is wrong!” – Is it? (https://eerielinux.wordpress.com/2017/11/25/permissive-licensing-is-wrong-is-it-1-2/)
A few weeks ago I’ve been attacked by some GNU zealots on a German tech site after speaking in favor of permissive licenses. Unfortunately a discussion was not possible there because that would require the will to actually communicate instead of simply accusing the other side of vile motives. Since I actually do care about this topic and a reader asked for a post about it in comments a while ago, here we go.
This first part tries to sum up the most important things around the topic. I deliberately aim for an objective overview that tries not to be one-sided. The second part will then contain my points in defence of permissive licensing.
Why license software at all?
Licenses exist for reasons of protection. If you’re the author/inventor of some software, a story or whatever product, you get to decide what to do with it. You can keep it for yourself or you can give it away. If you decide for the latter, you have to decide who may use it and in which way(s). In case you intend to give it to a (potentially) large group of people, you may not want to be asked for permission to xyz by everybody. That’s when you decide to write a license which states what you are allowing and explicitly disallowing.
Most of the well-known commercial licenses focus on what you’re not allowed to do (usually things like copying, disassembling, etc.). Open source licenses on the other hand are meant to grant the user rights (e.g. the right to distribute) while reserving some rights or only giving permission under certain conditions – and they usually make you claim responsibility for using the software. For these reasons licenses can actually be a good thing!
If you got an unlicensed piece of code, you’re not legally allowed to do anything with it without getting the author’s permission first. And even if you got that permission, your project would be risky, since the author can withdraw it later. A proper license protects both parties. The author doesn’t get his mail account full of email asking for permission, he’s save from legal trouble if his code breaks anything for you and at the same time you have legal certainty when you decide to put the code to long-term use.
Permissive vs. Copyleft (in a nutshell)
In short terms, permissive licensing usually goes like this: “Here you are, have fun. Oh, and don’t sue me if it does something else than what you expect!” Yes, it’s that easy and there’s little to dispute over.
Copyleft on the other side sounds like this (if you ask somebody in favor of Copyleft): “Sure, you can use it, it’s free. Just keep it free, ok?”. Also quite simple. And not too bad, eh? Other people however read the same thing like this: “Yes, you’re free to use it. Just read these ten pages of legalese and be dead certain that you comply. If you got something wrong, we will absolutely make you regret it.”
The GNU Public license (GPL)
The most popular copyleft license in use is the GPL (in various versions) (https://www.gnu.org/licenses/gpl.html). It got more and more complex with each version – and to be fair, it had to, because it was necessary to react to new threats and loop holes that were found later. The GNU project states that they are committed to protect what they call the four freedoms of free software:
the freedom to use the software for any purpose
the freedom to change the software to suit your needs
the freedom to share the software with your friends and neighbors
the freedom to share the changes you make
These are freedoms that every supporter of open source software should be able to agree with. So what’s the deal with all the hostility and fighting between the two camps? Let’s take a look at a permissive license, too.
The BSD license
Unlike the GPL, the BSD family of licenses begun with a rather simple license that span four rules (“original BSD license”). It was later revised and reduced to three (“modified BSD license”). And the modern BSD license that e.g. FreeBSD uses is even just two (“simplified BSD license”).
Did you read the GPLv3 that I linked to above? If you are using GPL’d code you really should. In case you don’t feel like reading all of it, at least take a look and grasp how long that text is. Now compare it to the complete modern BSD license (https://opensource.org/licenses/bsd-license.php).
What’s the problem?
There are essentially two problems that cause all the trouble. The first one is the question of what should be subject to the freedom that we’re talking about. And closely related, the second one is where that freedom needs to end.
Ironically both camps claim that freedom is the one important thing and it must not be restricted. The GPL is meant to protect the freedom of the software and enforces the availability of the source code, hence limiting the freedom of actual persons. BSD on the other hand is meant to protect the freedom of human beings who should be able to use the software as they see fit – even if that means closing down former open source code!
The GNU camp taunts permissive licenses as being “lax” for not providing the protection that they want. The other camp points out that the GPL is a complex monster and that it is virulent in nature: Since it’s very strict in a lot of areas, it’s incompatible with many other licenses. This makes it complicated to mix GPL and non-GPL code and in the cases where it’s legally possible, the GPL’s terms will take precedence and necessarily be in effect for the whole combined work.
Who’s right?
That totally depends on what you want to achieve. There are pros and cons to both – and in fact we’re only looking at the big picture here. There’s also e.g. the Apache license which is often deemed as kind of middle ground. Then you may want to consider the difference between weak (e.g. LGPL) as well as strong copyleft (GPL). Licensing is a potentially huge topic. But let’s keep it simple here because the exact details are actually not necessary to understand the essence of our topic.
In the next post I’ll present my stance on why permissive licensing is a good thing and copyleft is more problematic than many people may think.
“Permissive licensing is wrong?” – No it’s not! (https://eerielinux.wordpress.com/2018/01/25/permissive-licensing-is-wrong-no-its-not-2-2/)
The previous post gave a short introduction into the topic of software licenses, focusing on the GPL vs. BSD discussion. This one is basically my response to some typical arguments I’ve seen from people who seem to loathe permissive licensing. I’ll write this in dialog style, hoping that this makes it a little lighter to read.
Roundup
Install OpenBSD on dedibox with full-disk encryption (https://poolp.org/posts/2018-01-29/install-openbsd-on-dedibox-with-full-disk-encryption/)
TL;DR:
I run several "dedibox" servers at online.net, all powered by OpenBSD. OpenBSD is not officially supported so you have to work-around. Running full-disk encrypted OpenBSD there is a piece of cake. As a bonus, my first steps within a brand new booted machine ;-)
Step #0: choosing your server
OpenBSD is not officially supported, I can’t guarantee that this will work for you on any kind of server online.net provides, however I’ve been running https://poolp.org on OpenBSD there since 2008, only switching machines as they were getting a bit old and new offers came up.
Currently, I’m running two SC 2016 (SATA) and one XC 2016 (SSD) boxes, all three running OpenBSD reliably ever since I installed them.
Recently I’ve been willing to reinstall the XC one after I did some experiments that turned it into a FrankenBSD, so this was the right occasion to document how I do it for future references.
I wrote an article similar to this a few years ago relying on qemu to install to the disk, since then online.net provided access to a virtual serial console accessed within the browser, making it much more convenient to install without the qemu indirection which hid the NIC devices and disks duid and required tricks.
The method I currently use is a mix and adaptation from the techniques described in https://www.2f30.org/guides/openbsd-dedibox.html to boot the installer, and the technique described in https://geekyschmidt.com/2011/01/19/configuring-openbsd-softraid-fo-encryption.html to setup the crypto slice.
Step #1: boot to rescue mode
Step #2: boot to the installer
Step #3: prepare softraid
Step #4: reboot to encrypted OpenBSD system
Bonus: further tightening your system
enable doas
disable the root account
update system with syspatch
add my ssh public key to my ~/.ssh/authorized_keys
disable password authentication within ssh
reboot so you boot on a brand new up-to-date system with latest stable kernel
VOILA !
January 2018 Development Projects Update (https://www.freebsdfoundation.org/blog/january-2018-development-projects-update/)
Spectre and Meltdown in FreeBSD
Issues affecting most CPUs used in servers, desktops, laptops, and mobile devices are in the news. These hardware vulnerabilities, known by the code-names “Meltdown” and “Spectre”, allow malicious programs to read data to which they should not have access. This potentially includes credentials, cryptographic material, or other secrets. They were originally identified by a researcher from Google’s Project Zero, and were also independently discovered by researchers and academics from Cyberus Technology, Graz University of Technology, the University of Pennsylvania, the University of Maryland, Rambus, the University of Adelaide and Data61.
These vulnerabilities affect many CPU architectures supported by FreeBSD, but the 64-bit x86 family of processors from Intel and AMD are the most widely used, and are a high priority for software changes to mitigate the effects of Meltdown and Spectre. In particular, the Meltdown issue affects Intel CPUs and may be used to extract secret data from the running kernel, and therefore, is the most important issue to address.
The FreeBSD Foundation collaborates with Intel, and under this relationship participated in a briefing to understand the details of these issues and plan the mitigations to be applied to the x86 architectures supported by FreeBSD. We also made arrangements to have FreeBSD’s security officer join me in the briefing. It is through the generous support of the Foundation’s donors that we are able to dedicate resources to focus on these issues on demand as they arise.
Foundation staff member Konstantin (Kostik) Belousov is an expert on FreeBSD’s Virtual Memory (VM) system as well as low-level x86 details, and is developing the x86 kernel mitigations for FreeBSD.
The mitigation for Meltdown is known as Page Table Isolation (PTI). Kostik created a PTI implementation which was initially committed in mid-January and is available in the FreeBSD-CURRENT development repository. This is the same approach used by the Linux kernel to mitigate Meltdown.
One of the drawbacks of the PTI mitigation is that it incurs a performance regression. Kostik recently reworked FreeBSD’s use of Process-Context Identifiers (PCID) in order to regain some of the performance loss incurred by PTI. This change is also now available in FreeBSD-CURRENT.
The issue known as Spectre comes in two variants, and variant 2 is the more troubling and pressing one. It may be mitigated in one of two ways: by using a technique called “retpoline” in the compiler, or by making use of a CPU feature introduced in a processor microcode update. Both options are under active development. Kostik’s change to implement the CPU-based mitigation is currently in review. Unfortunately, it introduces a significant performance penalty and alternatives are preferred, if available.
For most cases, the compiler-based retpoline mitigation is likely to be the chosen mitigation. Having switched to the Clang compiler for the base system and most of the ports collection some years ago, FreeBSD is well-positioned to deploy Clang-based mitigations. FreeBSD developer Dimitry Andric is spearheading the update of Clang/LLVM in FreeBSD to version 6.0 in anticipation of its official release; FreeBSD-CURRENT now includes an interim snapshot. I have been assisting with the import, particularly with respect to LLVM’s lld linker, and will support the integration of retpoline. This support is expected to be merged into FreeBSD in the coming weeks.
The Foundation’s co-op students have also participated in the response to these vulnerabilities. Mitchell Horne developed the patch to control the PTI mitigation default setting, while Arshan Khanifar benchmarked the performance impact of the in-progress mitigation patches. In addition, Arshan and Mitchell each developed changes to FreeBSD’s tool chain to support the full set of mitigations that will be applied.
These mitigations will continue be tested, benchmarked, and refined in FreeBSD-CURRENT before being merged into stable branches and then being made available as updates to FreeBSD releases. Details on the timing of these merges and releases will be shared as they become available.
I would like to acknowledge all of those in the FreeBSD community who have participated in FreeBSD’s response to Meltdown and Spectre, for testing, reviewing, and coordinating x86 mitigations, for developing mitigations for other processor architectures and for the Bhyve hypervisor, and for working on the toolchain-based mitigations.
Guides: Getting Started & Lumina Theme Submissions (https://lumina-desktop.org/guides-getting-started-lumina-themes/)
I am pleased to announce the beginning of a new sub-series of blog posts for the Lumina project: Guides!
The TrueOS/Lumina projects want to support our users as they use Lumina or experiment with TrueOS. To that end, we’ve recently set up a central repository for our users to share instructions or other “how-to” guides with each other! Project developers and contributors will also submit guides to the repository on occasion, but the overall goal is to provide a simple hub for instructions written by any Lumina or TrueOS user. This will make it easier for users to not only find a “how-to” for some procedure, but also a very easy way to “give back” to the community by writing simple instructions or more detailed guides.
Guides Repository
Our first guide to get the whole thing started was created by the TrueOS Linebacker (https://discourse.trueos.org/t/introducing-the-trueos-linebacker/991) (with technical assistance from our own q5sys). In this guide, Terry Tate will walk you through the steps necessary to submit new wallpaper images to the Lumina Themes collection. This procedure is fully documented with screenshots every step of the way, walking you through a simple procedure that only requires a web browser and a Github account!
Guide: Lumina Themes Submissions (https://github.com/trueos/guides/blob/master/lumina-themes-submissions/readme.md)
The end result of this guide was that Terry Tate was able to submit this cool new “Lunar-4K” wallpaper to the “lumina-nature” collection.
TrueOS Community Guides (https://github.com/trueos/guides/tree/master)
ZFS vs. OpenZFS (by Michael Dexter) (https://www.ixsystems.com/blog/zfs-vs-openzfs/)
You’ve probably heard us say a mix of “ZFS” and “OpenZFS” and an explanation is long-overdue. Our Senior Analyst clears up what ZFS and OpenZFS refer to and how they differ.
I admit that we geeks tend to get caught up in the nuts and bolts of enterprise storage and overlook the more obvious questions that users might have. You’ve probably noticed that this blog and the FreeNAS blog refer to “ZFS” and “OpenZFS” seemingly at random when talking about the amazing file system at the heart of FreeNAS and every storage product that iXsystems sells. I will do my best to clarify what exactly these two terms refer to.
From its inception, “ZFS” has referred to the “Zettabyte File System” developed at Sun Microsystems and published under the CDDL Open Source license in 2005 as part of the OpenSolaris operating system. ZFS was revolutionary for completely decoupling the file system from specialized storage hardware and even a specific computer platform. The portable nature and advanced features of ZFS led FreeBSD, Linux, and even Apple developers to start porting ZFS to their operating systems and by 2008, FreeBSD shipped with ZFS in the 7.0 release. For the first time, ZFS empowered users of any budget with enterprise-class scalability and data integrity and management features like checksumming, compression and snapshotting, and those features remain unrivaled at any price to this day. On any ZFS platform, administrators use the zpool and zfs utilities to configure and manage their storage devices and file systems respectively. Both commands employ a user-friendly syntax such as‘zfs create mypool/mydataset’ and I welcome you to watch the appropriately-titled webinar “Why we love ZFS & you should too” or try a completely-graphical ZFS experience with FreeNAS.
Yes, ZFS is really as good as people say it is.
After enjoying nearly a decade of refinement by a growing group of developers around the world, ZFS became the property of database vendor Oracle, which ceased public development of both ZFS and OpenSolaris in 2010. Disappointed but undeterred, a group of OpenSolaris users and developers forked the last public release of OpenSolaris as the Illumos project. To this day, Illumos represents the official upstream home of the Open Source OpenSolaris technologies, including ZFS. The Illumos project enjoys healthy vendor and user participation but the portable nature and compelling features of ZFS soon produced far more ZFS users than Illumos users around the world. While most if not all users of Illumos and its derivatives are ZFS users, the majority of ZFS users are not Illumos users, thanks significantly in part to FreeNAS which uses the FreeBSD operating system. This imbalance plus several successful ZFS Day events led ZFS co-founder Matt Ahrens and a group of ZFS developers to announce the OpenZFS project, which would remain a part of the Illumos code base but would be free to coordinate development efforts and events around their favorite file system. ZFS Day has grown into the two-day OpenZFS Developer Summit and is stronger than ever, a testament to the passion and dedication of the OpenZFS community.
Oracle has steadily continued to develop its own proprietary branch of ZFS and Matt Ahrens points out that over 50% of the original OpenSolaris ZFS code has been replaced in OpenZFS with community contributions. This means that there are, sadly, two politically and technologically-incompatible branches of “ZFS” but fortunately, OpenZFS is orders of magnitude more popular thanks to its open nature. The two projects should be referred to as “Oracle ZFS” and “OpenZFS” to distinguish them as development efforts, but the user still types the ‘zfs’ command, which on FreeBSD relies on the ‘zfs.ko’ kernel module. My impression is that the terms of the CDDL license under which the OpenZFS branch of ZFS is published protects its users from any patent and trademark risks. Hopefully, this all helps you distinguish the OpenZFS project from the ZFS technology.
Beastie Bits
Explaining Shell (https://explainshell.com/)
OPNsense® 18.1 Released (https://opnsense.org/opnsense-18-1-released/)
“SSH Mastery 2/e” copyedits back (https://blather.michaelwlucas.com/archives/3104)
Sponsoring a Scam (https://blather.michaelwlucas.com/archives/3106)
Thursday, February 8, 2018 - Come to Netflix to talk about FreeBSD (https://www.meetup.com/BAFUG-Bay-Area-FreeBSD-User-Group/events/246623825/)
BSD User Group meeting in Stockholm: March 22, 17:30 - 21:00 (https://www.meetup.com/BSD-Users-Stockholm/events/247552279/)
FreeBSD Flavoured talks from Linux.conf.au: You can’t unit test C, right? (https://www.youtube.com/watch?v=z-uWt5wVVkU) and A Brief History of I/O (https://www.youtube.com/watch?v=qAhZEI_6lbc)
EuroBSDcon 2018 website is up (https://2018.eurobsdcon.org/)
Full day bhyvecon Tokyo, Japan, March 9, 2018 (http://bhyvecon.org/)
***
Feedback/Questions
Thomas - freebsd installer improvements (http://dpaste.com/3G2F7RC#wrap)
Mohammad - FreeBSD 11 installation from a read only rescue disk (http://dpaste.com/0HGK3FQ#wrap)
Stan - Follow up on guide you covered (http://dpaste.com/2S169SH#wrap)
Jalal - couple questions (http://dpaste.com/35N8QXP#wrap)
| |||
| 19 Aug 2015 | 103: Ubuntu Slaughters Kittens | 02:00:27 | |
Allan's away at BSDCam this week, but we've still got an exciting episode for you. We sat down with Bryan Cantrill, CTO of Joyent, to talk about a wide variety of topics: dtrace, ZFS, pkgsrc, containers and much more. This is easily our longest interview to date!
This episode was brought to you by
Interview - Bryan Cantrill - bryan@joyent.com (mailto:bryan@joyent.com) / @bcantrill (https://twitter.com/bcantrill)
BSD and Solaris history, illumos, dtrace, Joyent, pkgsrc, various topics (and rants)
Feedback/Questions
Randy writes in (http://slexy.org/view/s2b6dA7fAr)
Jared writes in (http://slexy.org/view/s2vABMHiok)
Steve writes in (http://slexy.org/view/s2194ADVUL)
***
| |||
| 17 Feb 2016 | 129: Synthesize all the Things! | 01:43:21 | |
Coming up this week, we will be talking to John Marino about his work on the ports-mgmt utility “Synth” and the cross-pollination between DragonFly and FreeBSD. That plus the latest news and your email here on
This episode was brought to you by
Headlines
glibc and the BSDs (https://blog.des.no/2016/02/freebsd-and-cve-2015-7547/)
You have likely already heard about CVE-2015-7547 (https://access.redhat.com/security/cve/cve-2015-7547)
“A stack-based buffer overflow was found in the way the libresolv library performed dual A/AAAA DNS queries. A remote attacker could create a specially crafted DNS response which could cause libresolv to crash or,
potentially, execute code with the permissions of the user running the library.”
“Note: this issue is only exposed when libresolv is called from the nss_dns NSS service module.”
More details from Google’s Online Security team blog (https://googleonlinesecurity.blogspot.ca/2016/02/cve-2015-7547-glibc-getaddrinfo-stack.html)
“Naturally, people have started asking whether FreeBSD is affected. The FreeBSD Security Officer has not yet released an official statement, but in the meantime, here is a brief look at the issue as far as FreeBSD is concerned.”
“First of all: neither FreeBSD itself nor native FreeBSD applications are affected. While the resolver in FreeBSD’s libc and GNU libc share a common parentage, the bug was introduced when the latter was rewritten to send A and AAAA queries in parallel rather than sequentially when the application requests both.”
The same most likely applies to the other BSDs
“However, Linux applications running under emulation on a FreeBSD system use the GNU libc and are therefore vulnerable unless patched.”
A patch to update emulation/linux_base-c6 has been prepared and should be committed soon
Running ‘pkg audit’ will list any known vulnerable packages installed on your system
“The issue can be mitigated by only using resolvers you trust, and configuring them to avoid sending responses which can trigger the bug.”
“If you already have your own resolvers, you can configure them to avoid sending UDP responses larger than 2048 bytes. If the response does not fit in 2048 bytes, the server will send a truncated response, and the client should retry using TCP. While a similar bug exists in the code path for TCP requests, I believe that it can only be exploited by a malicious resolver, and interposing your own resolver will protect affected Linux systems and applications.”
Dag-Erling’s blog post also includes instructions and configuration examples for locking down your resolver, or setting up your own resolver if you don’t have one already
***
OpenBSD Foundation - 2016 Fundraising Campaign (http://www.openbsdfoundation.org/campaign2016.html)
The OpenBSD foundation has announced their 2016 fundraising campaign, and set the goal of raising $250k for the year.
While they mention that fundraising for 2015 didn’t hit 2014’s blockbuster numbers, it still exceeded the goal set, with an almost equal mix of corporate and community donors.
‘Our goal for 2016 is to increase the amount of support we offer for development, without compromising our regular support for the projects. We would like to:
Plan and support more developer events (hackathons), and allow for more developers to attend these events.
Continue to improve the project infrastructure.
Fund more dedicated developer time for targeted development of specific projects.‘
To give you an idea of how much OpenBSD technology is used around the world, they broke it down this way:
If $10 were given for every installation of OpenBSD in the last year from the master site (ignoring the mirrors) we would be at our goal.
If $2 were given for every download of the OpenSSH source code in the last year from the master site (ignoring the mirrors) we would be at our goal.
If a penny was donated for every pf or OpenSSH installed with a mainstream operating system or phone in the last year we would be at our goal.
Getting Started with ION-DTN 3.4.0 on FreeBSD (https://sgeos.github.io/freebsd/ion/dtn/2016/02/07/getting-started-with-ion-dtn-3-4-0-on-freebsd.html)
“The Interplanetary Overlay Network (ION) software distribution is an implementation of Delay-Tolerant Networking (DTN) architecture as described in Internet RFC 4838, suitable for use in spacecraft”
This tutorial covers setting up ION 3.4.0 on FreeBSD
The tutorial starts by downloading the ION software, and installing the relevant build tools
The instructions allow ION to be installed system-wide, or for a specific user
The each host is configured
Then pings are traded between the hosts to ensure everything works
Then a web page is served over the interplanetary network
Sadly I don’t have any hosts on other planets to test with.
The tutorial also includes a troubleshooting guide
***
Open Storage Issue – New BSD Mag is Out! (https://bsdmag.org/download/open_storage/)
The next issue of BSDMag (The Open Storage Issue) just landed which features an interview with Matt Olander of iXsystems.
During the interview, Matt talks about the culture of support for open-source down at iX, not only FreeNAS and PC-BSD, but the FreeBSD foundation, Slackware and more.
He also gets to extol the virtues of the open-source development model itself, why it tends to lead to better code overall.
In addition to the lead interview with Matt, this issue also features some other great interviews with Open Source storage vendors, and even some ZFS howto’s about setting up your ZIL devive
***
Interview - John Marino - marino@freebsd.org (mailto:marino@freebsd.org)
FreeNAS with FreeBSD as its base helped save taxpayers $36,000 for a small public school district (https://www.ixsystems.com/whats-new/2016/02/11/january-missioncomplete-best-story/)
News Roundup
Getting Started With Tor Hidden Services on FreeBSD (https://sgeos.github.io/tor/freebsd/nc/curl/2016/02/06/getting-started-with-tor-hidden-services-on-freebsd.html)
Ever wondered how to setup and use a Tor hidden service? We have a walkthrough posted over on github.io which details how to do that on a FreeBSD -CURRENT system.
The basics are pretty simple, installing security/tor is the first step (although, he is using portmaster, you may wish to just ‘pkg install security/tor’)
The walkthrough provides an example server hosting just the date/time on port 8080, which you can use as an example and to verify it works, before serving anything real.
Once a local server is ready to serve something, the Tor setup is pretty quick, basically just two lines of config in torrc:
HiddenServiceDir /usr/home/tor/hidden_service/
HiddenServicePort 80 127.0.0.1:8080
After starting the service, the walkthrough will show you how to get the new hostname for this hidden service and verify its functionality.
ZFS Remote Mirrors for Home Use (https://github.com/hughobrien/zfs-remote-mirror)
A recently updated tutorial on remotely mirroring your ZFS files
Using a spare old computer, or a SBC like a Raspberry Pi, and an (external) hard drive
It covers installing and configuring FreeBSD for both sides of the remote replication
The new appendix covers the creation of a Raspberry Pi image, although a prebuilt one is also provided
The setup uses GELI to ensure the data is encrypted at-rest
Updating and maintaining both systems is covered in detail
The article is very detailed, and covers pretty much every aspect of the setup, including suggestions on where to physically locate the remote system, and configuration tips to reduce the chance that local intervention will be required
Most importantly, it covers the disaster recovery steps. How to get your files back when bad things happen
***
Lumina Desktop 0.8.8 Released (http://lumina-desktop.org/lumina-desktop-0-8-8-released/)
PC-BSD’s very own Lumina desktop has issued a new release, 0.8.8
Notable in this release is support for NetBSD out of box, improvements to the start menu, and ability to change monitor resolutions in the X configuration tool. (Also the desktop font colors look better!)
0.8.8 is now available in PC-BSD via pkg, and FreeBSD ports/pkg system as well.
Lumina Desktop aims for v1.0 in July 2016 (http://fossforce.com/2016/02/lumina-desktop-getting-ready-freebsd-11-0/)
We also have a blog post from Larry over at FossForce, highlighting that 1.0 of Lumina is still targeted for July(ish)
***
NetBSD on Google's Compute Engine (http://www.feyrer.de/NetBSD/bx/blosxom.cgi/nb_20160213_1951.html)
A NetBSD developer has gotten NetBSD running on Google Compute Engine, a service somewhat similar to Amazon’s EC2, and Microsoft’s Azure
Support is still being worked on, but I imagine it will land in NetBSD before too long
NetBSD on GCE dmesg (http://dmesgd.nycbug.org/index.cgi?action=dmesgd&do=view&id=2900)
OpenBSD on GCE (http://marc.info/?l=openbsd-misc&m=138610199311393&w=2)
FreeBSD on GCE (https://github.com/swills/FreeBSD-gcloud)
***
BeastieBits
htop 2.0 released - an interactive process viewer for Unix (including FreeBSD and OpenBSD) (http://hisham.hm/htop/)
Full set of binary packages for 7.0 released for ARM v6 and v7 (hf) (http://mail-index.netbsd.org/port-arm/2016/01/31/msg003648.html)
DragonFly 4.4.2 released (https://www.dragonflybsd.org/release44/)
LibertyBSD 5.8 has been released (http://libertybsd.net/)
Broadwell systems may want to take advantage of the patch by Imre Vadasz (http://lists.dragonflybsd.org/pipermail/commits/2016-January/459239.html)
Finding the hard-to-spot bugs in FreeBSD (http://www.viva64.com/en/b/0377/)
Feedback/Questions
Johnny - The Daily Show (http://slexy.org/view/s21dwzoXRn)
Randy - Let it BSD (http://slexy.org/view/s2Hmmu5pUr)
Miguel - NullFS (http://slexy.org/view/s20tOLsHHj)
Jaek - PC-BSD Hardware (http://slexy.org/view/s2N9wQ1n5X)
***
| |||
| 03 Oct 2024 | 579: EuroBSDcon 2024 | 00:56:36 | |
Jason is on location at EuroBSDcon getting interviews with those in the BSD Community.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Interviews
Vanja Cvelbar
Stefano Marinelli
Dave Cottlehuber
Christos Margiolis
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
Jason is on location at EuroBSDcon getting interviews with those in the BSD Community.
| |||
| 22 May 2019 | 299: The NAS Fleet | 00:52:47 | |
Running AIX on QEMU on Linux on Windows, your NAS fleet with TrueCommand, Unleashed 1.3 is available, LLDB: CPU register inspection support extension, V7 Unix programs often not written as expected, and more.
HeadlinesRunning AiX on QEMU on Linux on Windows
Take Command of Your NAS Fleet with TrueCommand
News RoundupUnleashed 1.3 Released
LLDB: extending CPU register inspection support
V7 Unix programs are often not written the way you would expect
Beastie Bits
Feedback/Questions
| |||
| 06 Jul 2016 | 149: The bhyve has been disturbed, and a wild Dexter appears! | 02:20:43 | |
Today on the show, we are going to be chatting with Michael Dexter about a variety of topics, but of course including bhyve! That plus
This episode was brought to you by
Headlines
NetBSD Introduction (https://bsdmag.org/netbsd_intr/)
We start off today’s episode with a great new NetBSD article!
Siju Oommen George has written an article for BSDMag, which provides a great overview of NetBSD’s beginnings and what it is today.
Of course you can’t start an article about NetBSD without mentioning where the name came from:
“The four founders of the NetBSD project, Chris Demetriou, Theo de Raadt, Adam Glass, and Charles Hannum, felt that a more open development model would benefit the project: one centered on portable, clean and correct code. They aimed to produce a unified, multi-platform, production-quality, BSD-based operating system. The name “NetBSD” was suggested by de Raadt, based on the importance and growth of networks, such as the Internet at that time, the distributed and collaborative nature of its development.”
From there NetBSD has expanded, and keeping in line with its motto “Of course it runs NetBSD” it has grown to over 57 hardware platforms, including “IA-32, Alpha, PowerPC,SPARC, Raspberry pi 2, SPARC64 and Zaurus”
From there topics such as pkgsrc, SMP, embedded and of course virtualization are all covered, which gives the reader a good overview of what to expect in the modern NetBSD today.
Lastly, in addition to mentioning some of the vendors using NetBSD in a variety of ways, including Point-Of-Sale systems, routers and thin-clients, you may not have known about the research teams which deploy NetBSD:
NASA Lewis Research Center – Satellite Networks and Architectures Branch use NetBSD almost exclusively in their investigation of TCP for use in satellite networks.
KAME project – A research group for implementing IPv6, IPsec and other recent TCP/IP related technologies into BSD UNIX kernels, under BSD license.
NEC Europe Ltd. established the Network Laboratories in Heidelberg, Germany in 1997, as NEC’s third research facility in Europe. The Heidelberg labs focus on software-oriented research and development for the next generation Internet.
SAMS-II Project – Space Acceleration Measurement System II. NASA will be measuring the microgravity environment on the International Space Station using a distributed system, consisting of NetBSD.“
My condolences, you’re now the maintainer of a popular open source project (https://runcommand.io/2016/06/26/my-condolences-youre-now-the-maintainer-of-a-popular-open-source-project/)
A presentation from a Wordpress conference, about what it is like to be the maintainer of a popular open source project
The presentation covers the basics:
Open Source is more than just the license, it is about community and involvement
The difference between Maintainers and Contributors
It covers some of the reasons people do not open up their code, and other common problems people run into:
“I'm embarrassed by my code” (Hint: so is everyone else, post it anyway, it is the best way to learn)
“I'm discouraged that I can't finish releases on time”
“I'm overwhelmed by the PR backlog”
“I'm frustrated when issues turn into flamewars”
“I'm overcommitted on my open source involvement”
“I feel all alone”
Each of those points is met with advice and possible solutions
So, there you have it. Open up your code, or join an existing project and help maintain it
***
FreeBSD Committer Allan Jude Discusses the Advantages of FreeBSD and His Role in Keeping Millions of Servers Running (http://www.hostingadvice.com/blog/freebsd-project-under-the-hood/)
An interesting twist on our normal news-stories today, we have an article featuring our very own Allan Jude, talking about why FreeBSD and the advantages of working on an open-source project.
“When Allan started his own company hosting websites for video streaming, FreeBSD was the only operating system he had previously used with other hosts. Based on his experience and comfort with it, he trusted the system with the future of his budding business.A decade later, the former-SysAdmin went to a conference focused on the open-source operating system, where he ran into some of the folks on its documentation team. “They inspired me,” he told our team in a recent chat. He began writing documentation but soon wanted to contribute improvements beyond the docs.Today, Allan sits as a FreeBSD Project Committer. It’s rare that you get to chat with someone involved with a massive-scale open-source project like this — rare and awesome.”
From there Allan goes into some of the reasons “Why” FreeBSD, starting with Code Organization being well-maintained and documented:
“The FreeBSD Project functions like an extremely well-organized world all its own. Allan explained the environment: “There’s a documentation page that explains how the file system’s laid out and everything has a place and it always goes in that place.””
+ In addition, Allan gives us some insight into his work to bring Boot-Environments to the loader, and other reasons why FreeBSD “just makes sense”
+ In summary Allan wraps it up quite nicely:
“An important take-away is that you don’t have to be a major developer with tons of experience to make a difference in the project,” Allan said — and the difference that devs like Allan are making is incredible. If you too want to submit the commit that contributes to the project relied on by millions of web servers, there are plenty of ways to get involved!
We’re especially talking to SysAdmins here, as Allan noted that they are the main users of FreeBSD. “Having more SysAdmins involved in the actual build of the system means we can offer the tools they’re looking for — designed the way a SysAdmin would want them designed, not necessarily the way a developer would think makes the most sense”
A guide to saving electricity and time with poudriere and bhyve (http://justinholcomb.me/blog/2016/07/03/poudriere-in-bhyve-and-bare-metal.html)
“This article goes over running poudriere to built packages for a Raspberry Pi with the interesting twist of running it both as a bhyve guest and then switching to running on bare metal via Fiber Channel via ctld by sharing the same ZFS volume.”
“Firstly, poudriere can build packages for different architectures such as ARM. This can save hours of build time compared to building ports from said ARM device.”
“Secondly, let’s say a person has an always-on device (NAS) running FreeBSD. To save power, this device has a CPU with a low clock-rate and low core count. This low clock-rate and core count is great for saving power but terrible for processor intensive application such as poudriere. Let’s say a person also has another physical server with fast processors and a high CPU count but draws nearly twice the power and a fan noise to match.”
“To get the best of both worlds, the goal is to build the packages on the fast physical server, power it down, and then start the same ZFS volume in a bhyve environment to serve packages from the always-on device.”
The tutorial walks through setting up ‘ahost’, the always on machine, ‘fhost’ the fast but noisy build machine, and a raspberry pi
It also includes creating a zvol, configuring iSCSI over fibre channel and exporting the zvol, booting an iSCSI volume in bhyve, plus installing and setting up poudriere
This it configures booting over fibre channel, and cross-building armv6 (raspberry pi) packages on the fast build machine
Then the fast machine is shut down, and the zvol is booted in bhyve on the NAS
Everything you need to know to make a hybrid physical/virtual machine
The same setup could also work to run the same bhyve VM from either ahost or fhost
bhyve does not yet support live migration, but when it does, having common network storage like the zvol will be an important part of that
***
Interview - Michael Dexter - editor@callfortesting.org (mailto:editor@callfortesting.org) / @michaeldexter (https://twitter.com/michaeldexter)
The RoloDexter
***
iXSystems
Children's Minnesota Star Studio Chooses iXsystems' TrueNAS Storage (https://www.youtube.com/watch?v=FFbdQ_05e-0)
***
News Roundup
FreeBSD Foundation June 2016 Update (https://www.freebsdfoundation.org/wp-content/uploads/2016/06/FreeBSD-Foundation-June-2016-Update.pdf)
The FreeBSD Foundation’s June newsletter is out
Make sure you submit the FreeBSD Community Survey (https://www.surveymonkey.com/r/freebsd2016) by July 7th:
In addition to the opening message from the executive director of the foundation, the update includes details to sponsored work on the FreeBSD VM system, reports from a number of conferences the Foundation attended, including BSDCan
The results of the foundation's yearly board meeting
People the foundation recognized for their contributions to FreeBSD at BSDCan
And an introduction to their new “Getting Started with FreeBSD” project
***
[How-To] Building the FreeBSD OS from scratch (http://www.all-nettools.com/forum/showthread.php?34422-Building-the-FreeBSD-OS-from-scratch)
A tutorial over at the All-NetTools.com forums that walks through building FreeBSD from scratch
I am not sure why anyone would want to build Xorg from source, but you can
It covers everything in quite a bit of detail, from the installation process through adding Xorg and a window manager from source
It also includes tweaking some device node permissions for easier operation as a non-root user, and configuring the firewall
***
Window Systems Should Be Transparent (http://doc.cat-v.org/bell_labs/transparent_wsys/) + Rob Pike of AT&T Labs writes about why Window Systems should be transparent
This is an old paper (undated, but I think from the late 80s), but may contain some timeless insights
“UNIX window systems are unsatisfactory. Because they are cumbersome and complicated, they are unsuitable companions for an operating system that is appreciated for its technical elegance”
“A good interface should clarify the view, not obscure it”
“Mux is one window system that is popular and therefore worth studying as an example of good design. (It is not commercially important because it runs only on obsolete hardware.) This paper uses mux as a case study to illustrate some principles that can help keep a user interface simple, comfortable, and unobtrusive. When designing their products, the purveyors of commercial window systems should keep these principles in mind.”
There are not many commercial window systems anymore, but “open source” was not really a big thing when this paper was written
***
Roger Faulkner, of Solaris fame passed away (http://permalink.gmane.org/gmane.comp.standards.posix.austin.general/12877)
“RIP Roger Faulkner: creator of the One and True /proc, slayer of the M-to-N threading model -- and the godfather of post-AT&T Unix”
@bcantrill: Another great Roger Faulkner story (https://twitter.com/bcantrill/status/750442169807171584)
The story of how pgrep -w saved a monitor -- if not a life (https://news.ycombinator.com/item?id=4306515)
@bcantrill: With Roger Faulkner, Tim led an engineering coup inside Sun that saved Solaris circa 2.5 (https://twitter.com/bcantrill/status/750442169807171584)
***
Beastie Bits:
Developer Ed Maste is requesting information from those who are users of libvgl. (https://lists.freebsd.org/pipermail/freebsd-stable/2016-June/084843.html)
HEADS UP: DragonFly 4.5 world reneeds rebuilding (http://lists.dragonflybsd.org/pipermail/users/2016-June/249748.html)
Chris Buechler is leaving the pfSense project, the entire community thanks you for your many years of service (https://blog.pfsense.org/?p=2095)
GhostBSD 10.3-BETA1 now available (http://ghostbsd.org/10.3_BETA1)
DragonFlyBSD adds nvmectl (http://lists.dragonflybsd.org/pipermail/commits/2016-June/500671.html)
OPNsense 16.1.18 released (https://opnsense.org/opnsense-16-1-18-released/)
bhyve_graphics hit CURRENT (https://svnweb.freebsd.org/base?view=revision&revision=302332)
BUG Update
FreeBSD Central Twitter account looking for a new owner (https://twitter.com/freebsdcentral/status/750053703420350465)
NYCBUG meeting : Meet the Smallest BSDs: RetroBSD and LiteBSD, Brian Callahan (http://lists.nycbug.org/pipermail/talk/2016-July/016732.html)
NYCBUG install fest @ HOPE (http://lists.nycbug.org/pipermail/talk/2016-June/016694.html)
SemiBUG is looking for presentations for September and beyond (http://lists.nycbug.org/pipermail/semibug/2016-June/000107.html)
Caleb Cooper is giving a talk on Crytpo at KnoxBUG on July 26th (http://knoxbug.org/content/2016-07-26)
Feedback/Questions
Leif - ZFS xfer (http://pastebin.com/vvASr64P)
Zach - Python3 (http://pastebin.com/SznQHq7n)
Dave - Versioning (http://pastebin.com/qkpjKEr0)
David - Encrypted Disk Images (http://pastebin.com/yr7BUmv2)
Eli - TLF in all the wrong places (http://pastebin.com/xby81NvC)
***
| |||
| 23 Apr 2020 | 347: New Directions | 01:00:50 | |
Rethinking OpenBSD security, FreeBSD 2020 Q1 status report, the notion of progress and user interfaces, Comments about Thomas E. Dickey on NetBSD curses, making Unix a little more Plan9-like, Not-actually Linux distro review: FreeBSD, and more.
Headlines
Rethinking OpenBSD Security (https://flak.tedunangst.com/post/rethinking-openbsd-security)
OpenBSD aims to be a secure operating system. In the past few months there were quite a few security errata, however. That’s not too unusual, but some of the recent ones were a bit special. One might even say bad. The OpenBSD approach to security has a few aspects, two of which might be avoiding errors and minimizing the risk of mistakes. Other people have other ideas about how to build secure systems. I think it’s worth examining whether the OpenBSD approach works, or if this is evidence that it’s doomed to failure.
I picked a few errata, not all of them, that were interesting and happened to suit my narrative.
FreeBSD 2020 Q1 Quarterly report (https://www.freebsd.org/news/status/report-2020-01-2020-03.html)
Welcome, to the quarterly reports, of the future! Well, at least the first quarterly report from 2020. The new timeline, mentioned in the last few reports, still holds, which brings us to this report, which covers the period of January 2020 - March 2020.
News Roundup
The Notion of Progress and User Interfaces (https://herebeseaswines.net/essays/2020-04-13-the-notion-of-progress-and-user-interfaces)
One trait of modern Western culture is the notion of progress. A view claiming, at large, everything is getting better and better.
How should we think about progress? Both in general and regarding technology?
Thomas E. Dickey on NetBSD curses (https://implementality.blogspot.com/2020/04/thomas-e-dickey-on-netbsd-curses.html)
I was recently pointed at a web page on Thomas E. Dickeys site talking about NetBSD curses. It seems initially that the page was intended to be a pointer to some differences between ncurses and NetBSD curses and does appear to start off in this vein but it seems that the author has lost the plot as the document evolved and the tail end of it seems to be devolving into some sort of slanging match. I don't want to go through Mr. Dickey's document point by point, that would be tedious but I would like to pick out some of the things that I believe to be the most egregious. Please note that even though I am a NetBSD developer, the opinions below are my own and not the NetBSD projects.
Making Unix a little more Plan9-like (https://woozle.org/papers/plan9.html)
I’m not really interested in defending anything. I tried out plan9port and liked it, but I have to live in Unix land. Here’s how I set that up.
A Warning
The suckless community, and some of the plan9 communities, are dominated by jackasses. I hope that’s strong enough wording to impress the severity. Don’t go into IRC for help. Stay off the suckless email list. The software is great, the people who write it are well-spoken and well-reasoned, but for some reason the fandom is horrible to everyone.
Not-actually Linux distro review: FreeBSD 12.1-RELEASE (https://arstechnica.com/gadgets/2020/04/not-actually-linux-distro-review-freebsd-12-1-release/)
This month's Linux distro review isn't of a Linux distribution at all—instead, we're taking a look at FreeBSD, the original gangster of free Unix-like operating systems.
The first FreeBSD release was in 1993, but the operating system's roots go further back—considerably further back. FreeBSD started out in 1992 as a patch-release of Bill and Lynne Jolitz's 386BSD—but 386BSD itself came from the original Berkeley Software Distribution (BSD). BSD itself goes back to 1977—for reference, Linus Torvalds was only seven years old then.
Before we get started, I'd like to acknowledge something up front—our distro reviews include the desktop experience, and that is very much not FreeBSD's strength. FreeBSD is far, far better suited to running as a headless server than as a desktop! We're going to get a full desktop running on it anyway, because according to Lee Hutchinson, I hate myself—and also because we can't imagine readers wouldn't care about it.
FreeBSD does not provide a good desktop experience, to say the least. But if you're hankering for a BSD-based desktop, don't worry—we're already planning a followup review of GhostBSD, a desktop-focused BSD distribution.
Beastie Bits
Wifi renewal restarted (https://blog.netbsd.org/tnf/entry/wifi_renewal_restarted)
HAMMER2 and a quick start for DragonFly (https://www.dragonflydigest.com/2020/04/21/24421.html)
Engineering NetBSD 9.0 (http://netbsd.org/~kamil/AsiaBSDCon/Kamil_Rytarowski_Engineering_NetBSD_9.0.pdf)
Antivirus Protection using OPNsense Plugins (https://www.youtube.com/watch?v=94vz_-5lAkE)
BSDCan Home Lab Panel recording session: May 5th at 18:00 UTC (https://twitter.com/allanjude/status/1251895348836143104)
BSDNow is going Independent
After being part of Jupiter Broadcasting since we started back in 2013, BSDNow is moving to become independent. We extend a very large thank you to Jupiter Broadcasting and Linux Academy for hosting us for so many years, and allowing us to bring you over 100 episodes without advertisements. LinuxAcademy is now under new leadership, and we understand that cutbacks needed to be made, and that BSD is not their core product. That does not mean your favourite BSD podcast is going away, we will continue and we expect things will not look much different.
What does this mean for you, the listener? Not much will change, just make sure your subscription is via the RSS feed at BSDNow.tv rather than one of the Jupiter Broadcasting feeds. We will update you with more news as things settle out.
Feedback/Questions
Jordyn - ZFS Pool Problem (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/347/feedback/Jordyn%20zfs%20pool%20problem.md)
debug - https://github.com/BSDNow/bsdnow.tv/raw/master/episodes/347/feedback/dbg.txt
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 21 Mar 2024 | 551: SSH Port Story | 00:52:21 | |
This week on the show, The story of SSH getting port 22, GGC using Clang, AuxRunner, Stabweek, Using a Kensington SlimBladePro on OpenBSD, and more...
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
The story of getting SSH port 22 (https://www.ssh.com/academy/ssh/port#the-story-of-getting-ssh-port-22)
Can GCC use Clang as its assembler? (https://briancallahan.net/blog/20240122.html)
News Roundup
AUXrunner: a macOS QEMU-based app for running A/UX (https://mendelson.org/auxrunner.html)
Stabweek (https://lists.freebsd.org/archives/freebsd-current/2024-February/005657.html)
Using the Kensington SlimBlade Pro TrackBall with OpenBSD (https://www.tumfatig.net/2024/using-the-kensington-slimblade-pro-trackball-with-openbsd/)
Running 9front on an emulated SGI Indy via MAME (https://posixcafe.org/blogs/2024/01/01/0/)
Beastie Bits
Huffman Codes – How Do They Work? (https://two-wrongs.com/huffman-codes-how-do-they-work)
NetBSD 10.0_RC5 (https://mail-index.netbsd.org/source-changes/2024/02/27/msg150156.html)
New code for SIGILL faults help identify misbranches (https://www.undeadly.org/cgi?action=article;sid=20240222183703)
New Illumos telegram channel (https://t.me/illumosDistroes)
The Jan Feb issues of the FreeBSD Journal is here (https://freebsdfoundation.org/blog/the-january-february-2024-issue-of-the-freebsd-journal-is-here/)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 26 Jan 2023 | 491: Catch the Spammers | 00:42:18 | |
Dragonfly BSD 6.4 is out, Running OpenZFS – Choosing Between FreeBSD and Linux, OpenBSD Mastery: Filesystems ebook leaks, catching 71% spam, crazy unix shell prompts, Linux Binary Compatibility: Ubuntu on FreeBSD, Reproducible Builds Summit Venice 2022, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Dragonfly BSD 6.4 is out (https://www.dragonflybsd.org/release64/)
Running OpenZFS – Choosing Between FreeBSD and Linux (https://klarasystems.com/articles/running-openzfs-choosing-between-freebsd-and-linux/)
News Roundup
“OpenBSD Mastery: Filesystems” ebook leaking out (https://mwl.io/archives/22462)
Can Your Spam-eater Manage to Catch Seventy-one Percent Like This Other Service? (https://bsdly.blogspot.com/2022/12/can-your-spam-eater-manage-to-catch.html)
Crazy unix shell prompts (https://lists.nycbug.org:8443/pipermail/semibug/2022-December/000775.html)
Linux Binary Compatibility: Ubuntu on FreeBSD (https://byte--sized-de.translate.goog/linux-unix/linux-binary-compatibility-ubuntu-unter-freebsd/?_x_tr_sl=auto&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp)
Reproducible Builds Summit Venice 2022 (https://blog.netbsd.org/tnf/entry/reproducible_builds_summit_venice_2022)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Felix - Managing Jails with ansible (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/491/feedback/Felix%20-%20Managing%20Jails%20with%20ansible.md)
John Baldwin - bhyve networking setup article (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/491/feedback/John%20Baldwin%20-%20bhyve%20networking%20setup%20article.md)
Welton - bhyve webadmin (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/491/feedback/Welton%20-%20bhyve%20webadmin.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 06 May 2015 | 88: Below the Clouds | 01:34:00 | |
This time on the show, we'll be talking with Ed Schouten about CloudABI. It's a new application binary interface with a strong focus on isolation and restricted capabilities. As always, all this week's BSD news and answers to your emails, on BSD Now - the place to B.. SD.
This episode was brought to you by
Headlines
FreeBSD quarterly status report (https://www.freebsd.org/news/status/report-2015-01-2015-03.html)
The FreeBSD team has posted a report of the activities that went on between January and March of this year
As usual, it's broken down into separate reports from the various teams in the project (ports, kernel, virtualization, etc)
The ports team continuing battling the flood of PRs, closing quite a lot of them and boasting nearly 7,000 commits this quarter
The core team and cluster admins dealt with the accidental deletion of the Bugzilla database, and are making plans for an improved backup strategy within the project going forward
FreeBSD's future release support model was also finalized and published in February, which should be a big improvement for both users and the release team
Some topics are still being discussed internally, mainly MFCing ZFS ARC responsiveness patches to the 10 branch and deciding whether to maintain or abandon C89 support in the kernel code
Lots of activity is happening in bhyve, some of which we've covered recently (http://www.bsdnow.tv/episodes/2015_04_29-on_the_list), and a number of improvements were made this quarter
Clang, LLVM and LLDB have been updated to the 3.6.0 branch in -CURRENT
Work to get FreeBSD booting natively on the POWER8 CPU architecture is also still in progress, but it does boot in KVM for the time being
The project to replace forth in the bootloader with lua is in its final stages, and can be used on x86 already
ASLR work (http://www.bsdnow.tv/episodes/2014_08_27-reverse_takeover) is still being done by the HardenedBSD guys, and their next aim is position-independent executable
The report also touches on multipath TCP support, the new automounter, opaque ifnet, pkgng updates, secureboot (which should be in 10.2-RELEASE), GNOME and KDE on FreeBSD, PCIe hotplugging, nested kernel support and more
Also of note: work is going on to make ARM a Tier 1 platform in the upcoming 11.0-RELEASE (and support for more ARM boards is still being added, including ARM64)
***
OpenBSD 5.7 released (http://www.openbsd.org/57.html)
OpenBSD has formally released another new version, complete with the giant changelog we've come to expect
In the hardware department, 5.7 features many driver improvements and fixes, as well as support for some new things: USB 3.0 controllers, newer Intel and Atheros wireless cards and some additional 10gbit NICs
If you're using one of the Soekris boards, there's even a new driver (http://bodgitandscarper.co.uk/openbsd/further-soekris-net6501-improvements-for-openbsd/) to manipulate the GPIO and LEDs on them - this has some fun possibilities
Some new security improvements include: SipHash (https://en.wikipedia.org/wiki/SipHash) being sprinkled in some areas to protect hashing functions, big W^X improvements (https://www.marc.info/?l=openbsd-tech&m=142120787308107&w=2) in the kernel space, static PIE (http://www.bsdnow.tv/episodes/2015_04_15-pie_in_the_sky) on all architectures, deterministic "random" functions being replaced (https://www.marc.info/?l=openbsd-tech&m=141807224826859&w=2) with strong randomness, and support for remote logging over TLS
The entire source tree has also been audited to use reallocarray (http://lteo.net/blog/2014/10/28/reallocarray-in-openbsd-integer-overflow-detection-for-free/), which unintentionally saved (https://splone.com/blog/2015/3/11/integer-overflow-prevention-in-c) OpenBSD's libc from being vulnerable to earlier attacks (https://guidovranken.wordpress.com/2015/02/04/full-disclosure-heap-overflow-in-h-spencers-regex-library-on-32-bit-systems/) affecting other BSDs' implementations
Being that it's OpenBSD, a number of things have also been removed from the base system: procfs, sendmail, SSLv3 support and loadable kernel modules are all gone now (not to mention the continuing massacre of dead code in LibreSSL)
Some people seem to be surprised about the removal of loadable modules, but almost nothing utilized them in OpenBSD, so it was really just removing old code that no one used anymore - very different from FreeBSD or Linux in this regard, where kernel modules are used pretty heavily
BIND and nginx have been taken out, so you'll need to either use the versions in ports or switch to Unbound and the in-base HTTP daemon
Speaking of httpd, it's gotten a number of new (http://www.openbsd.org/papers/httpd-slides-asiabsdcon2015.pdf) features (http://www.openbsd.org/cgi-bin/man.cgi/OpenBSD-current/man5/httpd.conf.5), and has had time to grow and mature since its initial debut - if you've been considering trying it out, now would be a great time to do so
This release also includes the latest OpenSSH (with stronger fingerprint types and host key rotation), OpenNTPD (with the HTTPS constraints feature), OpenSMTPD, LibreSSL and mandoc (http://www.bsdnow.tv/episodes/2014_11_12-a_mans_man)
Check the errata page (http://www.openbsd.org/errata57.html) for any post-release fixes, and the upgrade guide (http://www.openbsd.org/faq/upgrade57.html) for specific instructions on updating from 5.6
Groundwork has also been laid for some major SMP scalability improvements - look forward to those in future releases
There's a song and artwork (http://www.openbsd.org/lyrics.html#57) to go along with the release as always, and CDs should be arriving within a few days - we'll show some pictures next week
Consider picking one up (https://www.openbsdstore.com) to support the project (and it's the only way to get puffy stickers)
For those of you paying close attention, the banner image (http://www.openbsd.org/images/puffy57.gif) for this release just might remind you of a certain special episode (http://www.bsdnow.tv/episodes/2014_09_03-its_hammer_time) of BSD Now...
***
Tor-BSD diversity project (https://torbsd.github.io/)
We've talked about Tor on the show a few times, and specifically about getting more of the network on BSD (Linux has an overwhelming majority right now)
A new initiative has started to do just that, called the Tor-BSD diversity project
"Monocultures in nature are dangerous, as vulnerabilities are held in common across a broad spectrum. Diversity means single vulnerabilities are less likely to harm the entire ecosystem. [...] A single kernel vulnerability in GNU/Linux that impacting Tor relays could be devastating. We want to see a stronger Tor network, and we believe one critical ingredient for that is operating system diversity."
In addition to encouraging people to put up more relays, they're also continuing work on porting the Tor Browser Bundle to BSD, so more desktop users can have easy access to online privacy
There's an additional progress report (http://trac.haqistan.net/blog/tor-browser-ports-progress) for that part specifically, and it looks like most of the work is done now
Engaging the broader BSD community about Tor and fixing up the official documentation are also both on their todo list
If you've been considering running a node to help out, there's always our handy tutorial (http://www.bsdnow.tv/tutorials/tor) on getting set up
***
PC-BSD 10.1.2-RC1 released (http://blog.pcbsd.org/2015/05/pc-bsd-10-1-2-rc1-now-available/)
If you want a sneak peek at the upcoming PC-BSD 10.1.2, the first release candidate is now available to grab
This quarterly update includes a number of new features, improvements and even some additional utilities
PersonaCrypt is one of them - it's a new tool for easily migrating encrypted home directories between systems
A new "stealth mode" option allows for a one-time login, using a blank home directory that gets wiped after use
Similarly, a new "Tor mode" allows for easy tunneling of all your traffic through the Tor network
IPFW is now the default firewall, offering improved VIMAGE capabilities
The life preserver backup tool now allows for bare-metal restores via the install CD
ISC's NTP daemon has been replaced with OpenNTPD (http://www.bsdnow.tv/episodes/2015_02_11-time_for_a_change), and OpenSSL has been replaced with LibreSSL (http://www.bsdnow.tv/episodes/2015_03_25-ssl_in_the_wild)
It also includes the latest Lumina (http://www.bsdnow.tv/episodes/2014_09_10-luminary_environment) desktop, and there's another post dedicated to that (http://blog.pcbsd.org/2015/05/pc-bsd-10-1-2-rc1-lumina-desktop-0-8-4-released/)
Binary packages have also been updated to fresh versions from the ports tree
More details, including upgrade instructions, can be found in the linked blog post
***
Interview - Ed Schouten - ed@freebsd.org (mailto:ed@freebsd.org) / @edschouten (https://twitter.com/edschouten)
CloudABI (https://www.bsdcan.org/2015/schedule/track/Security/524.en.html)
News Roundup
Open Household Router Contraption (http://code.saghul.net/index.php/2015/05/01/announcing-the-open-household-router-contraption/)
This article introduces OpenHRC, the "Open Household Router Contraption"
In short, it's a set of bootstrapping scripts to turn a vanilla OpenBSD install into a feature-rich gateway device
It also makes use of Ansible playbooks for configuration, allowing for a more "mass deployment" type of setup
Everything is configured via a simple text file, and you end up with a local NTP server, DHCP server, firewall (obviously) and local caching DNS resolver - it even does DNSSEC validation
All the code is open source and on Github (https://github.com/ioc32/openhrc), so you can read through what's actually being changed and put in place
There's also a video guide (https://www.youtube.com/watch?v=LZeKDM5jc90) to the entire process, if you're more of a visual person
***
OPNsense 15.1.10 released (https://forum.opnsense.org/index.php?topic=365.0)
Speaking of BSD routers, if you're looking for a "prebuilt and ready to go" option, OPNsense has just released a new version
15.1.10 drops some of the legacy patches they inherited from pfSense, aiming to stay closer to the mainline FreeBSD source code
Going along with this theme, they've redone how they do ports, and are now kept totally in sync with the regular ports tree
Their binary packages are now signed using the fingerprint-style method, various GUI menus have been rewritten and a number of other bugs were fixed
NanoBSD-based images are also available now, so you can try it out on hardware with constrained resources as well
Version 15.1.10.1 (https://twitter.com/opnsense/status/596009164746432512) was released shortly thereafter, including a hotfix for VLANs
***
IBM Workpad Z50 and NetBSD (https://www.ibm.com/developerworks/community/blogs/hpcgoulash/entry/ibm_workpad_z50_netbsd_an_interesting_combination1?lang=en)
Before the infamous netbook fad came and went, IBM had a handheld PDA device that looked pretty much the same
Back in 1999, they released the Workpad Z50 (http://www.hpcfactor.com/reviews/hardware/ibm/workpad-z50/) with Windows CE, sporting a 131MHz MIPS CPU, 16MB of RAM and a 640x480 display
You can probably tell where this is going... the article is about installing NetBSD it
"What prevents me from taking my pristine Workpad z50 to the local electronics recycling facility is NetBSD. With a little effort it is possible to install recent versions of NetBSD on the Workpad z50 and even have XWindows running"
The author got pkgsrc up and running on it too, and cleverly used distcc to offload the compiling jobs to something a bit more modern
He's also got a couple (https://www.youtube.com/watch?v=hSLVnSZKB9I) videos (https://www.youtube.com/watch?v=mIA-NWEHLM4) of the bootup process and running Xorg (neither of which we'd call "speedy" by any stretch of the imagination)
***
FreeBSD from the trenches (http://freebsdfoundation.blogspot.com/2015/04/from-trenches-tips-tricks-edition.html)
The FreeBSD foundation has a new blog post up in their "from the trenches" series, detailing FreeBSD in some real-world use cases
In this installment, Glen Barber talks about how he sets up all his laptops with ZFS and GELI
While the installer allows for an automatic ZFS layout, Glen notes that it's not a one-size-fits-all thing, and goes through doing everything manually
Each command is explained, and he walks you through the process of doing an encrypted installation (http://www.bsdnow.tv/tutorials/fde) on your root zpool
***
Broadwell in DragonFly (http://lists.dragonflybsd.org/pipermail/users/2015-May/207671.html)
DragonFlyBSD has officially won the race to get an Intel Broadwell graphics driver
Their i915 driver has been brought up to speed with Linux 3.14's, adding not only Broadwell support, but many other bugfixes for other cards too
It's planned for commit to the main tree very soon, but you can test it out with a git branch for the time being
***
Feedback/Questions
Bostjan writes in (http://slexy.org/view/s216QQcHyX)
Hunter writes in (http://slexy.org/view/s21hGSk3c0)
Hrishi writes in (http://slexy.org/view/s20JwPw9Je)
Clint writes in (http://slexy.org/view/s2x1GYr7y6)
Sergei writes in (http://slexy.org/view/s2swXxr2PX)
***
Mailing List Gold
How did you guess (https://lists.freebsd.org/pipermail/freebsd-advocacy/2015-May/004541.html)
***
| |||
| 12 Aug 2021 | 415: Wrong OS Switch | 00:54:17 | |
Wrong Way to Switch Server OS, Net/1 and Net/2 – A Path to Freedom, Permissions Two Mistakes, OpenBSD progress in supporting riscv64 platform, I2P intro, git sync murder is out, GhostBSD init system poll, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow)
Headlines
The Wrong Way to Switch Operating Systems on Your Server (https://figbert.com/posts/wrong-way-to-switch-server-os/)
History of FreeBSD Part 5: Net/1 and Net/2 – A Path to Freedom (https://klarasystems.com/articles/history-of-freebsd-net-1-and-net-2-a-path-to-freedom/)
News Roundup
Permissions Two Mistakes (https://utcc.utoronto.ca/~cks/space/blog/unix/PermissionsTwoMistakes)
Progress in support for the riscv64 platform (https://undeadly.org/cgi?action=article;sid=20210619161607)
I2P Intro (https://dataswamp.org/~solene/2021-06-20-i2p-intro.html)
“$ git sync murder” is out, so: how many books have I written? (https://mwl.io/archives/12105)
What init system would you prefer to use under GhostBSD? (https://www.ghostbsd.org/what_init_system_pool)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Brad - Replication (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/415/feedback/Brad%20-%20Replication.md)
Benedict writes after the show was over: The tool is called https://github.com/allanjude/zxfer
Tom tweeted right after recording stopped:
https://twitter.com/adventureloop/status/1420478529238622210
Caleb - Pronunciation of Gemini (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/415/feedback/Caleb%20-%20Pronounciation%20of%20Gemini.md)
Dan - Writeup about a DO FreeBSD Droplet (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/415/feedback/Dan%20-%20Writeup%20about%20a%20DO%20FreeBSD%20Droplet.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 23 Jan 2025 | 595: Arc: the Triumph | 01:48:23 | |
Applying the ARC Algorithm to the ARC, Advancing Cloud Native Containers on FreeBSD: Podman Testing Highlights, Running Web Browsers in FreeBSD Jail, Fixing pf not allowing IPv6 traffic on FreeBSD, Minitel: The Online World France Built Before the Web, Why Google Stores Billions of Lines of Code in a Single Repository, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Applying the ARC Algorithm to the ARC (https://klarasystems.com/articles/applying-the-arc-algorithm-to-the-arc/?utm_source=BSD%20Now&utm_medium=Podcast)
Advancing Cloud Native Containers on FreeBSD: Podman Testing Highlights (https://freebsdfoundation.org/blog/advancing-cloud-native-containers-on-freebsd-podman-testing-highlights/)
News Roundup
Running Web Browsers in FreeBSD Jail (https://tumfatig.net/2024/running-web-browsers-in-freebsd-jail/)
Fixing pf not allowing IPv6 traffic on FreeBSD (https://www.ncartron.org/fixing-pf-not-allowing-ipv6-traffic-on-freebsd.html)
Minitel: The Online World France Built Before the Web (https://spectrum.ieee.org/minitel-the-online-world-france-built-before-the-web)
Why Google Stores Billions of Lines of Code in a Single Repository (https://cacm.acm.org/research/why-google-stores-billions-of-lines-of-code-in-a-single-repository/)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Sam - EDR Support (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/595/feedback/Sam%20-%20EDR%20Support.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 12 Mar 2020 | 341: U-NAS-ification | 00:51:01 | |
FreeBSD on Power, DragonflyBSD 5.8 is here, Unifying FreeNAS/TrueNAS, OpenBSD vs. Prometheus and Go, gcc 4.2.1 removed from FreeBSD base, and more.
Headlines
FreeBSD on Power (https://www.freebsdfoundation.org/blog/power-to-the-people-making-freebsd-a-first-class-citizen-on-power/)
The power and promise of all open source software is freedom. Another way to express freedom is choice — choice of platforms, deployment models, stacks, configurations, etc.
The FreeBSD Foundation is dedicated to supporting and promoting the FreeBSD Project and community worldwide. But, what does this mean, exactly, you may wonder. The truth is it means many different things, but in all cases the Foundation acts to expand freedom and choice so that FreeBSD users have the power to serve their varied compute needs.
This blog tells the story of one specific way the Foundation helps a member of the community provide greater hardware choice for all FreeBSD users.
Dragonfly 5.8 (https://www.dragonflybsd.org/release58/)
DragonFly version 5.8 brings a new dsynth utility for building your own binary dports packages, plus significant support work to speed up that build - up to and including the entire collection. Additional progress has been made on GPU and signal support.
The details of all commits between the 5.6 and 5.8 branches are available in the associated commit messages for 5.8.0rc1 and 5.8.0. Also see /usr/src/UPDATING for specific file changes in PAM.
See article for rest of information
2nd HamBUG meeting recap (https://www.hambug.ca/)
The second meeting of the Hamilton BSD Users Group took place last night
The next meeting is scheduled for the 2nd Tuesday of the month, April 14th 2020
News Roundup
FreeNAS/TrueNAS Brand Unification (https://www.ixsystems.com/blog/freenas-truenas-unification/)
FreeNAS and TrueNAS have been separate-but-related members of the #1 Open Source storage software family since 2012. FreeNAS is the free Open Source version with an expert community and has led the pursuit of innovations like Plugins and VMs. TrueNAS is the enterprise version for organizations of all sizes that need additional uptime and performance, as well as the enterprise-grade support necessary for critical data and applications.
From the beginning at iXsystems, we’ve developed, tested, documented, and released both as separate products, even though the vast majority of code is shared. This was a deliberate technical decision in the beginning but over time became less of a necessity and more of “just how we’ve always done it”. Furthermore, to change it was going to require a serious overhaul to how we build and package both products, among other things, so we continued to kick the can down the road. As we made systematic improvements to development and QA efficiency over the past few years, the redundant release process became almost impossible to ignore as our next major efficiency roadblock to overcome. So, we’ve finally rolled up our sleeves.
With the recent 11.3 release, TrueNAS gained parity with FreeNAS on features like VMs and Plugins, further homogenizing the code. Today, we announce the next phase of evolution for FreeNAS and TrueNAS.
OpenBSD versus Prometheus (and Go). (https://utcc.utoronto.ca/~cks/space/blog/sysadmin/OpenBSDVsPrometheusAndGo)
We have a decent number of OpenBSD machines that do important things (and that have sometimes experienced problems like running out of disk space), and we have a Prometheus based metrics and monitoring system. The Prometheus host agent has enough support for OpenBSD to be able to report on critical metrics, including things like local disk space. Despite all of this, after some investigation I've determined that it's not really sensible to even try to deploy the host agent on our OpenBSD machines. This is due to a combination of factors that have at their root OpenBSD's lack of ABI stability
FreeBSD removed gcc from base (https://svnweb.freebsd.org/base?view=revision&revision=358454)
As described in Warner's email message[1] to the FreeBSD-arch mailing list we have reached GCC 4.2.1's retirement date. At this time all supported architectures either use in-tree Clang, or rely on external toolchain (i.e., a contemporary GCC version from ports).
GCC 4.2.1 was released July 18, 2007 and was imported into FreeBSD later that year, in r171825. GCC has served us well, but version 4.2.1 is obsolete and not used by default on any architecture in FreeBSD. It does not support modern C and does not support arm64 or RISC-V.
Beastie Bits
New Archive location for Dragonfly 4.x (https://www.dragonflydigest.com/2020/03/10/24276.html)
A dead simple git cheat sheet (https://hub.iwebthings.com/a-dead-simple-git-cheatsheet/)
Xorg 1.20.7 on HardenedBSD Comes with IE/RELRO+BIND_NOW/CFI/SafeStack Protections (https://twitter.com/lattera/status/1233412881569415168)
Feedback/Questions
Niclas writes in Regarding the Lenovo E595 user (episode 340) (http://dpaste.com/2YJ6PFW#wrap)
Lyubomir writes about GELI and ZFS (http://dpaste.com/1S0DGT3#wrap)
Peter writes in about scaling FreeBSD jails (http://dpaste.com/2FSZQ8V#wrap)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 22 Apr 2015 | 86: Business as Usual | 01:44:14 | |
Coming up this time on the show, we'll be chatting with Antoine Jacoutot about how M:Tier uses BSD in their business. After that, we'll be discussing the different release models across the BSDs, and which style we like the most. As always, answers to your emails and all the latest news, on BSD Now - the place to B.. SD.
This episode was brought to you by
Headlines
Optimizing TLS for high bandwidth applications (https://people.freebsd.org/~rrs/asiabsd_2015_tls.pdf)
Netflix has released a report on some of their recent activities, pushing lots of traffic through TLS on FreeBSD
TLS has traditionally had too much overhead for the levels of bandwidth they're using, so this pdf outlines some of their strategy in optimizing it
The sendfile() syscall (which nginx uses) isn't available when data is encrypted in userland
To get around this, Netflix is proposing to add TLS support to the FreeBSD kernel
Having encrypted movie streams would be pretty neat
***
Crypto in unexpected places (https://www.marc.info/?l=openbsd-cvs&m=142944822223482&w=2)
OpenBSD is somewhat known for its integrated cryptography, right down to strong randomness in every place you could imagine (process IDs, TCP initial sequence numbers, etc)
One place you might not expect crypto to be used (or even needed) is in the "ping" utility, right? Well, think again
David Gwynne recently committed (https://www.marc.info/?l=openbsd-cvs&m=142944754923359&w=2) a change that adds MAC (https://en.wikipedia.org/wiki/Message_authentication_code) to the ping timestamp payload
By default, it'll be filled with a ChaCha stream instead of an unvarying payload, and David says "this lets us have some confidence that the timestamp hasn't been damaged or tampered with in transit"
Not only is this a security feature, but it should also help detect dodgy or malfunctioning network equipment going forward
Maybe we can look forward to a cryptographically secure "echo" command next...
***
Broadwell in DragonFly (http://www.dragonflybsd.org/docs/newhandbook/docs/newhandbook/BroadwellBoxes/)
The DragonFlyBSD guys have started a new page on their wiki to discuss Broadwell hardware and its current status
Matt Dillon, the project lead, recently bought some hardware with this chipset, and lays out what works and what doesn't work
The two main show-stoppers right now are the graphics and wireless, but they have someone who's already making progress with the GPU support
Wireless support will likely have to wait until FreeBSD gets it, then they'll port it back over
None of the BSDs currently have full Broadwell support, so stay tuned for further updates
***
DIY NAS software roundup (http://blog.brianmoses.net/2015/04/diy-nas-software-roundup.html)
In this blog post, the author compares a few different software solutions for a network attached storage device
He puts FreeNAS, one of our favorites, up against a number of opponents - both BSD and Linux-based
NAS4Free gets an honorable mention as well, particularly for its lower hardware requirements and sleek interface
If you've been thinking about putting together a NAS, but aren't quite comfortable enough to set it up by yourself yet, this article should give you a good view of the current big names
Some competition is always good, gotta keep those guys on their toes
***
Interview - Antoine Jacoutot - ajacoutot@openbsd.org (mailto:ajacoutot@openbsd.org) / @ajacoutot (https://twitter.com/ajacoutot)
OpenBSD at M:Tier (http://www.mtier.org/about-us/), business adoption of BSD, various topics
News Roundup
OpenBSD on DigitalOcean (http://www.tubsta.com/2015/04/openbsd-on-digital-ocean/)
When DigitalOcean rolled out initial support for FreeBSD, it was a great step in the right direction - we hoped that all the other BSDs would soon follow
This is not yet the case, but a blog article here has details on how you can install OpenBSD (and likely the others too) on your VPS
Using a -current snapshot and some swapfile trickery, it's possible to image an OpenBSD ramdisk installer onto an unmounted portion of the virtual disk
After doing so, you just boot from their web UI-based console and can perform a standard installation
You will have to pay special attention to some details of the disk layout, but this article takes you through the entire process step by step
***
Initial ARM64 support lands in FreeBSD (https://svnweb.freebsd.org/base?view=revision&revision=281494)
The ARM64 architecture, sometimes called ARMv8 or AArch64 (https://wiki.freebsd.org/arm64), is a new generation of CPUs that will mostly be in embedded devices
FreeBSD has just gotten support for this platform in the -CURRENT branch
Previously, it was only the beginnings of the kernel and enough bits to boot in QEMU - now a full build (https://lists.freebsd.org/pipermail/freebsd-testing/2015-April/000918.html) is possible
Work should now start happening in the main source code tree, and hopefully they'll have full support in a branch soon
***
Scripting with least privilege (http://shill.seas.harvard.edu/)
A new scripting language with a focus on privilege separation and running with only what's absolutely needed has been popular in the headlines lately
Shell scripts are used everywhere today: startup scripts, orchestration scripts for mass deployment, configuring and compiling software, etc.
Shill aims to answer the questions "how do we limit the authority of scripts" and "how do we determine what authority is necessary" by including a declarative security policy that's checked and enforced by the language runtime
If used on FreeBSD, Shill will use Capsicum for sandboxing
You can find some more of the technical information in their documentation pdf (http://shill.seas.harvard.edu/shill-osdi-2014.pdf) or watch their USENIX presentation (https://2459d6dc103cb5933875-c0245c5c937c5dedcca3f1764ecc9b2f.ssl.cf2.rackcdn.com/osdi14/moore.mp4) video
Hacker News also had some discussion (https://news.ycombinator.com/item?id=9328277) on the topic
***
OpenBSD first impressions (http://blog.greduan.com/2015-04-19-mstobfi.html)
A brand new BSD user has started documenting his experience through a series of blog posts
Formerly a Linux guy, he's tried out FreeBSD and OpenBSD so far, and is currently working on an OpenBSD desktop
The first post goes into why he chose BSD at all, why he's switching away from Linux, how the initial transition has been, what you'll need to relearn and what he's got planned going forward
He's only been using OpenBSD for a few days as of the time this was written - we don't usually get to hear from people this early in on their BSD journey, so it offers a unique perspective
***
PCBSD and 4K oh my! (http://blog.pcbsd.org/2015/04/pc-bsd-and-4k-oh-my/)
Yesterday, Kris got ahold of some 4K monitor hardware to test PC-BSD out
The short of it - It works great!
Minor tweaks being made to some of the PC-BSD defaults to better accommodate 4K out of box
This particular model monitor ships with DisplayPort set to 1.1 mode only, switching it to 1.2 mode enables 60Hz properly
***
Feedback/Questions
Darin writes in (http://slexy.org/view/s21kFuvAFs)
Mitch writes in (http://slexy.org/view/s2nf4o9p4E)
***
Discussion
Comparison of BSD release cycles
FreeBSD (https://www.freebsd.org/doc/en_US.ISO8859-1/books/faq/introduction.html#idp55486416), OpenBSD (http://www.openbsd.org/faq/faq5.html#Flavors), NetBSD (https://www.netbsd.org/releases/release-map.html) and DragonFlyBSD (https://www.dragonflybsd.org/releases/)
***
| |||
| 21 May 2020 | 351: Heaven: OpenBSD 6.7 | 00:49:09 | |
Backup and Restore on NetBSD, OpenBSD 6.7 available, Building a WireGuard Jail with FreeBSD's standard tools, who gets to chown things and quotas, influence TrueNAS CORE roadmap, and more.
Headlines
Backup and Restore on NetBSD (https://e17i.github.io/articles-netbsd-backup/)
Putting together the bits and pieces of a backup and restore concept, while not being rocket science, always seems to be a little bit ungrateful. Most Admin Handbooks handle this topic only within few pages. After replacing my old Mac Mini's OS by NetBSD, I tried to implement an automated backup, allowing me to handle it similarly to the time machine backups I've been using before. Suggestions on how to improve are always welcome.
BSD Release: OpenBSD 6.7 (https://distrowatch.com/?newsid=10921)
The OpenBSD project produces and operating system which places focus on portability, standardisation, code correctness, proactive security and integrated cryptography. The project's latest release is OpenBSD 6.7 which introduces several new improvements to the cron scheduling daemon, improvements to the web server daemon, and the top command now offers scrollable output. These and many more changes can be found in the project's release announcement: "This is a partial list of new features and systems included in OpenBSD 6.7. For a comprehensive list, see the changelog leading to 6.7. General improvements and bugfixes: Reduced the minimum allowed number of chunks in a CONCAT volume from 2 to 1, increasing the number of volumes which can be created on a single disk with bioctl(8) from 7 to 15. This can be used to create more partitions than previously. Rewrote the cron(8) flag-parsing code to be getopt-like, allowing tight formations like -ns and flag repetition. Renamed the 'options' field in crontab(5) to 'flags'. Added crontab(5) -s flag to the command field, indicating that only a single instance of the job should run concurrently. Added cron(8) support for random time values using the ~ operator. Allowed cwm(1) configuration of window size based on percentage of the master window during horizontal and vertical tiling actions."
Release Announcement (https://marc.info/?l=openbsd-announce&m=158989783626149&w=2)
Release Notes (https://www.openbsd.org/67.html)
News Roundup
Building a WireGuard Jail with the FreeBSD's Standard Tools (https://genneko.github.io/playing-with-bsd/networking/freebsd-wireguard-jail/)
Recently, I had an opportunity to build a WireGuard jail on a FreeBSD 12.1 host.
As it was really quick and easy to setup and it has been working completely fine for a month, I’d like to share my experience with anyone interested in this topic.
The Unix divide over who gets to chown things, and (disk space) quotas (https://utcc.utoronto.ca/~cks/space/blog/unix/ChownDivideAndQuotas)
One of the famous big splits between the BSD Unix world and the System V world is whether ordinary users can use chown (the command and the system call) to give away their own files. In System V derived Unixes you were generally allowed to; in BSD derived Unixes you weren't. Until I looked it up now to make sure, I thought that BSD changed this behavior from V7 and that V7 had an unrestricted chown. However, this turns out to be wrong; in V7 Unix, chown(2) was restricted to root only.
You Can Influence the TrueNAS CORE Roadmap! (https://www.ixsystems.com/blog/truenas-bugs-and-suggestions/)
As many of you know, we’ve historically had three ticket types available in our tracker: Bugs, Features, and Improvements, which are all fairly self-explanatory. After some discussion internally, we’ve decided to implement a new type of ticket, a “Suggestion”. These will be replacing Feature and Improvement requests for the TrueNAS Community, simplifying things down to two options: Bugs and Suggestions. This change also introduces a slightly different workflow than before.
Beastie Bits
FreeNAS Spare Parts Build: Testing ZFS With Imbalanced VDEVs and Mismatched Drives (https://www.youtube.com/watch?v=EFrlG3CUKFQ)
TLSv1.3 server code enabled in LibreSSL in -current (https://undeadly.org/cgi?action=article;sid=20200512074150)
Interview with Deb Goodkin (https://itsfoss.com/freebsd-interview-deb-goodkin/)
***
Feedback/Questions
Bostjan - WireGaurd (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/351/feedback/Bostjan%20-%20WireGaurd.md)
Chad - ZFS Pool Design (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/351/feedback/Chad%20-%20ZFS%20Pool%20Design.md)
Pedreo - Scale FreeBSD Jails (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/351/feedback/Pedreo%20-%20Scale%20FreeBSD%20Jails.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 21 Dec 2016 | 173: Carry on my Wayland son | 01:14:38 | |
This week on the show, we’ve got some great stories to bring you, a look at the odder side of UNIX history
This episode was brought to you by
Headlines
syspatch in testing state (http://marc.info/?l=openbsd-tech&m=148058309126053&w=2)
Antoine Jacoutot ajacoutot@ openbsd has posted a call for testing for OpenBSD’s new syspatch tool
“syspatch(8), a "binary" patch system for -release is now ready for early testing. This does not use binary diffing to update the system, but regular signed tarballs containing the updated files (ala installer).”
“I would appreciate feedback on the tool. But please send it directly to me, there's no need to pollute the list. This is obviously WIP and the tool may or may not change in drastic ways.”
“These test binary patches are not endorsed by the OpenBSD project and should not be trusted, I am only providing them to get early feedback on the tool. If all goes as planned, I am hoping that syspatch will make it into the 6.1 release; but for it to happen, I need to know how it breaks your systems :-)”
Instructions (http://syspatch.openbsd.org/pub/OpenBSD/6.0/syspatch/amd64/README.txt)
If you test it, report back and let us know how it went
***
Weston working (https://lists.freebsd.org/pipermail/freebsd-current/2016-December/064198.html)
Over the past few years we’ve had some user-interest in the state of Wayland / Weston on FreeBSD. In the past day or so, Johannes Lundberg has sent in a progress report to the FreeBSD mailing lists.
Without further ADO:
We had some progress with Wayland that we'd like to share.
Wayland (v1.12.0)
Working
Weston (v1.12.0)
Working (Porting WIP)
Weston-clients (installed with wayland/weston port)
Working
XWayland (run X11 apps in Wayland compositor)
Works (maximized window only) if started manually but not when
launching X11 app from Weston. Most likely problem with Weston IPC.
Sway (i3-compatible Wayland compositor)
Working
SDL20 (Wayland backend)
games/stonesoup-sdl briefly tested.
https://twitter.com/johalun/status/811334203358867456
GDM (with Wayland)
Halted - depends on logind.
GTK3
gtk3-demo runs fine on Weston (might have to set GDK_BACKEND=wayland
first.
GTK3 apps working (gedit, gnumeric, xfce4-terminal tested, xfce desktop
(4.12) does not yet support GTK3)“
Johannes goes on to give instructions on how / where you can fetch their WiP and do your own testing. At the moment you’ll need Matt Macy’s newer Intel video work, as well as their ports tree which includes all the necessary software bits.
Before anybody asks, yes we are watching this for TrueOS!
***
Where the rubber meets the road (part two) (https://functionallyparanoid.com/2016/12/15/where-the-rubber-meets-the-road-part-two/)
Continuing with our story from Brian Everly from a week ago, we have an update today on the process to dual-boot OpenBSD with Arch Linux.
As we last left off, Arch was up and running on the laptop, but some quirks in the hardware meant OpenBSD would take a bit longer.
With those issues resolved and the HD seen again, the next issue that reared its head was OpenBSD not seeing the partition tables on the disk. After much frustration, it was time to nuke and pave, starting with OpenBSD first this time.
After a successful GPT partitioning and install of OpenBSD, he went back to installing Arch, and then the story got more interesting.
“I installed Arch as I detailed in my last post; however, when I fired up gdisk I got a weird error message:
“Warning! Disk size is smaller than the main header indicates! Loading secondary header from the last sector of the disk! You should use ‘v’ to verify disk integrity, and perhaps options on the expert’s menu to repair the disk.”
Immediately after this, I saw a second warning:
“Caution: Invalid backup GPT header, but valid main header; regenerating backup header from main header.”
And, not to be outdone, there was a third:
“Warning! Main and backup partition tables differ! Use the ‘c’ and ‘e’ options on the recovery & transformation menu to examine the two tables.”
Finally (not kidding), there was a fourth:
“Warning! One or more CRCs don’t match. You should repair the disk!”
Given all of that, I thought to myself, “This is probably why I couldn’t see the disk properly when I partitioned it under Linux on the OpenBSD side. I’ll let it repair things and I should be good to go.” I then followed the recommendation and repaired things, using the primary GPT table to recreate the backup one. I then installed Arch and figured I was good to go.“
After confirming through several additional re-installs that the behavior was reproducible, he then decided to go full on crazy,and partition with MBR. That in and of itself was a challenge, since as he mentions, not many people dual-boot OpenBSD with Linux on MBR, especially using luks and lvm!
If you want to see the details on how that was done, check it out.
The story ends in success though! And better yet:
“Now that I have everything working, I’ll restore my config and data to Arch, configure OpenBSD the way I like it and get moving. I’ll take some time and drop a note on the tech@ mailing list for OpenBSD to see if they can figure out what the GPT problem was I was running into. Hopefully it will make that part of the code stronger to get an edge-case bug report like this.”
Take note here, if you run into issues like this with any OS, be sure to document in detail what happened so developers can explore solutions to the issue.
***
FreeBSD and ZFS as a time capsule for OS X (https://blog.feld.me/posts/2016/12/using-freebsd-as-a-time-capsule-for-osx/)
Do you have any Apple users in your life? Perhaps you run FreeBSD for ZFS somewhere else in the house or office. Well today we have a blog post from Mark Felder which shows how you can use FreeBSD as a time-capsule for your OSX systems.
The setup is quite simple, to get started you’ll need packages for netatalk3 and avahi-app for service discovery.
Next up will be your AFP configuration. He helpfully provides a nice example that you should be able to just cut-n-paste. Be sure to check the hosts allow lines and adjust to fit your network. Also of note will be the backup location and valid users to adjust.
A little easier should be the avahi setup, which can be a straight copy-n-paste from the site, which will perform the service advertisements.
The final piece is just enabling specific services in /etc/rc.conf and either starting them by hand, or rebooting. At this point your OSX systems should be able to discover the new time-capsule provider on the network and DTRT.
***
News Roundup
netbenches - FreeBSD network forwarding performance benchmark results (https://github.com/ocochard/netbenches)
Olivier Cochard-Labbé, original creator of FreeNAS, and leader of the BSD Router Project, has a github repo of network benchmarks
There are many interesting results, and all of the scripts, documentation, and configuration files to run the tests yourself
IPSec Performance on an Atom C2558, 12-head vs IPSec Performance Branch (https://github.com/ocochard/netbenches/tree/master/Atom_C2558_4Cores-Intel_i350/ipsec/results/fbsd12.projects-ipsec.equilibrium)
Compared to: Xeon L5630 2.13GHz (https://github.com/ocochard/netbenches/tree/2f3bb1b3c51e454736f1fcc650c3328071834f8d/Xeon_L5630-4Cores-Intel_82599EB/ipsec/results/fbsd11.0)
and IPSec with Authentication (https://github.com/ocochard/netbenches/tree/305235114ba8a3748ad9681c629333f87f82613a/Atom_C2558_4Cores-Intel_i350/ipsec.ah/results/fbsd12.projects-ipsec.equilibrium)
I look forward to seeing tests on even more hardware, as people with access to different configurations try out these benchmarks
***
A tcpdump Tutorial and Primer with Examples (https://danielmiessler.com/study/tcpdump/)
Most users will be familiar with the basics of using tcpdump, but this tutorial/primer is likely to fill in a lot of blanks, and advance many users understanding of tcpdump
“tcpdump is the premier network analysis tool for information security professionals. Having a solid grasp of this über-powerful application is mandatory for anyone desiring a thorough understanding of TCP/IP. Many prefer to use higher level analysis tools such as Wireshark, but I believe this to usually be a mistake.”
tcpdump is an important tool for any system or network administrator, it is not just for security. It is often the best way to figure out why the network is not behaving as expected.
“In a discipline so dependent on a true understanding of concepts vs. rote learning, it’s important to stay fluent in the underlying mechanics of the TCP/IP suite. A thorough grasp of these protocols allows one to troubleshoot at a level far beyond the average analyst, but mastery of the protocols is only possible through continued exposure to them.”
Not just that, but TCP/IP is a very interesting protocol, considering how little it has changed in its 40+ year history
“First off, I like to add a few options to the tcpdump command itself, depending on what I’m looking at. The first of these is -n, which requests that names are not resolved, resulting in the IPs themselves always being displayed. The second is -X, which displays both hex and ascii content within the packet.”
“It’s also important to note that tcpdump only takes the first 96 bytes of data from a packet by default. If you would like to look at more, add the -s number option to the mix, where number is the number of bytes you want to capture. I recommend using 0 (zero) for a snaplength, which gets everything.”
The page has a nice table of the most useful options
It also has a great primer on doing basic filtering
If you are relatively new to using tcpdump, I highly recommend you spend a few minutes reading through this article
***
How Unix made it to the top (http://minnie.tuhs.org/pipermail/tuhs/2016-December/007519.html)
Doug McIlroy gives us a nice background post on how “Unix made it to the top”
It’s fairly short / concise, so I felt it would be good to read in its entirety.
“It has often been told how the Bell Labs law department became the first non-research department to use Unix, displacing a newly acquired stand-alone word-processing system that fell short of the department's hopes because it couldn't number the lines on patent applications, as USPTO required. When Joe Ossanna heard of this, he told them about roff and promised to give it line-numbering capability the next day. They tried it and were hooked. Patent secretaries became remote members of the fellowship of the Unix lab. In due time the law department got its own machine.
Less well known is how Unix made it into the head office of AT&T. It seems that the CEO, Charlie Brown, did not like to be seen wearing glasses when he read speeches. Somehow his PR assistant learned of the CAT phototypesetter in the Unix lab and asked whether it might be possible to use it to produce scripts in large type. Of course it was. As connections to the top never hurt, the CEO's office was welcomed as another ouside user. The cost--occasionally having to develop film for the final copy of a speech--was not onerous.
Having teethed on speeches, the head office realized that Unix could also be useful for things that didn't need phototypesetting. Other documents began to accumulate in their directory. By the time we became aware of it, the hoard came to include minutes of AT&T board meetings. It didn't seem like a very good idea for us to be keeping records from the inner sanctum of the corporation on a computer where most everybody had super-user privileges. A call to the PR guy convinced him of the wisdom of keeping such things on their own premises. And so the CEO's office bought a Unix system.
Just as one hears of cars chosen for their cupholders, so were theseusers converted to Unix for trivial reasons: line numbers and vanity.“
Odd Comments and Strange Doings in Unix (http://orkinos.cmpe.boun.edu.tr/~kosar/odd.html)
Everybody loves easter-eggs, and today we have some fun odd ones from the history throughout UNIX told by Dennis Ritchie.
First up, was a fun one where the “mv” command could sometimes print the following “values of b may give rise to dom!”
“Like most of the messages recorded in these compilations, this one was produced in some situation that we considered unlikely or as result of abuse; the details don't matter. I'm recording why the phrase was selected.
The very first use of Unix in the "real business" of Bell Labs was to type and produce patent applications, and for a while in the early 1970s we had three typists busily typing away in the grotty lab on the sixth floor. One day someone came in and observed on the paper sticking out of one of the Teletypes, displayed in magnificent isolation, this ominous phrase: values of b may give rise to dom!
It was of course obvious that the typist had interrupted a printout (generating the "!" from the ed editor) and moved up the paper, and that the context must have been something like "varying values of beta may give rise to domain wall movement" or some other fragment of a physically plausible patent application.But the phrase itself was just so striking! Utterly meaningless, but it looks like what... a warning? What is "dom?"
At the same time, we were experimenting with text-to-voice software by Doug McIlroy and others, and of course the phrase was tried out with it. For whatever reason, its rendition of "give rise to dom!" accented the last word in a way that emphasized the phonetic similarity between "doom" and the first syllable of "dominance." It pronounced "beta" in the British style, "beeta." The entire occurrence became a small, shared treasure.The phrase had to be recorded somewhere, and it was, in the v6 source. Most likely it was Bob Morris who did the deed, but it could just as easily have been Ken. I hope that your browser reproduces the b as a Greek beta.“
Next up is one you might have heard before:
/* You are not expected to understand this */> Every now and then on Usenet or elsewhere I run across a reference to a certain comment in the source code of the Sixth
Edition Unix operating system.
I've even been given two sweatshirts that quote it.
Most probably just heard about it, but those who saw it in the flesh either had Sixth Edition Unix (ca. 1975) or read the annotated version of this system by John Lions (which was republished in 1996: ISBN 1-57298-013-7, Peer-to-Peer Communications).It's often quoted as a slur on the quantity or quality of the comments in the Bell Labs research releases of Unix. Not an unfair observation in general, I fear, but in this case unjustified.
So we tried to explain what was going on. "You are not expected to understand this" was intended as a remark in the spirit of "This won't be on the exam," rather than as an impudent challenge.
There’s a few other interesting stories as well, if the odd/fun side of UNIX history at all interests you, I would recommend checking it out.
Beastie Bits
With patches in review the #FreeBSD base system builds 100% reproducibly (https://twitter.com/ed_maste/status/811289279611682816)
BSDCan 2017 Call for Participation (https://www.freebsdfoundation.org/news-and-events/call-for-papers/bsdcan-2017/)
ioCell 2.0 released (https://github.com/bartekrutkowski/iocell/releases)
who even calls link_ntoa? (http://www.tedunangst.com/flak/post/who-even-calls-link-ntoa)
Booting Androidx86 under bhyve (https://twitter.com/pr1ntf/status/809528845673996288)
Feedback/Questions
Chris - VNET (http://pastebin.com/016BfvU9)
Brian - Package Base (http://pastebin.com/8JJeHuRT)
Wim - TrueOS Desktop All-n-one (http://pastebin.com/VC0DPQUF)
Daniel - Long Boots (http://pastebin.com/q7pFu7pR)
Bryan - ZFS / FreeNAS (http://pastebin.com/xgUnbzr7)
Bryan - FreeNAS Security (http://pastebin.com/qqCvVTLB)
***
| |||
| 31 May 2017 | 196: PostgreZFS | 01:46:15 | |
This week on BSD Now, we review the EuroBSDcon schedule, we explore the mysteries of Docker on OpenBSD, and show you how to run PostgreSQL on ZFS.
This episode was brought to you by
Headlines
EuroBSDcon 2017 - Talks & Schedule published (https://2017.eurobsdcon.org/2017/05/26/talks-schedule-published/)
The EuroBSDcon website was updated with the tutorial and talk schedule for the upcoming September conference in Paris, France.
Tutorials on the 1st day: Kirk McKusick - An Introduction to the FreeBSD Open-Source Operating System, George Neville-Neil - DTrace for Developers, Taylor R Campbell - How to untangle your threads from a giant lock in a multiprocessor system
Tutorials on the 2nd day: Kirk continues his Introduction lecture, Michael Lucas - Core concepts of ZFS (half day), Benedict Reuschling - Managing BSD systems with Ansible (half day), Peter Hessler - BGP for developers and sysadmins
Talks include 3 keynotes (2 on the first day, beginning and end), another one at the end of the second day by Brendan Gregg
Good mixture of talks of the various BSD projects
Also, a good amount of new names and faces
Check out the full talk schedule (https://2017.eurobsdcon.org/talks-schedule/).
Registration is not open yet, but will be soon.
***
OpenBSD on the Xiaomi Mi Air 12.5" (https://jcs.org/2017/05/22/xiaomiair)
The Xiaomi Mi Air 12.5" (https://xiaomi-mi.com/notebooks/xiaomi-mi-notebook-air-125-silver/) is a basic fanless 12.5" Ultrabook with good build quality and decent hardware specs, especially for the money: while it can usually be had for about $600, I got mine for $489 shipped to the US during a sale about a month ago.
Xiaomi offers this laptop in silver and gold. They also make a 13" version but it comes with an NVidia graphics chip. Since these laptops are only sold in China, they come with a Chinese language version of Windows 10 and only one or two distributors that carry them ship to the US. Unfortunately that also means they come with practically no warranty or support.
Hardware
> The Mi Air 12.5" has a fanless, 6th generation (Skylake) Intel Core m3 processor, 4Gb of soldered-on RAM, and a 128Gb SATA SSD (more on that later). It has a small footprint of 11.5" wide, 8" deep, and 0.5" thick, and weighs 2.3 pounds.
> A single USB-C port on the right-hand side is used to charge the laptop and provide USB connectivity. A USB-C ethernet adapter I tried worked fine in OpenBSD. Whether intentional or not, a particular design touch I appreciated was that the USB-C port is placed directly to the right of the power button on the keyboard, so you don't have to look or feel around for the port when plugging in the power cable.
> A single USB 3 type-A port is also available on the right side next to the USB-C port. A full-size HDMI port and a headphone jack are on the left-hand side. It has a soldered-on Intel 8260 wireless adapter and Bluetooth. The webcam in the screen bezel attaches internally over USB.
> The chassis is all aluminum and has sufficient rigidity in the keyboard area. The 12.5" 1920x1080 glossy IPS screen has a fairly small bezel and while its hinge is properly weighted to allow opening the lid with one hand (if you care about that kind of thing), the screen does have a bit of top-end wobble when open, especially when typing on another laptop on the same desk.
> The keyboard has a roomy layout and a nice clicky tactile with good travel. It is backlit, but with only one backlight level. When enabled via Fn+F10 (which is handled by the EC, so no OpenBSD support required), it will automatically shut off after not typing for a short while, automatically turning back once a key is pressed.
Upgrades
> An interesting feature of the Mi Air is that it comes with a 128Gb SATA SSD but also includes an open PCI-e slot ready to accept an NVMe SSD.
> I upgraded mine with a Samsung PM961 256Gb NVMe SSD (left), and while it is possible to run with both drives in at the same time, I removed the Samsung CM871a 128Gb SATA (right) drive to save power.
> The bottom case can be removed by removing the seven visible screws, in addition to the one under the foot in the middle back of the case, which just pries off. A spudger tool is needed to release all of the plastic attachment clips along the entire edge of the bottom cover.
> Unfortunately this upgrade proved to be quite time consuming due to the combination of the limited UEFI firmware on the Mi Air and a bug in OpenBSD.
A Detour into UEFI Firmware Variables
> Unlike a traditional BIOS where one can boot into a menu and configure the boot order as well as enabling and disabling options such as "USB Hard Drive", the InsydeH2O UEFI firmware on the Xiaomi Air only provides the ability to adjust the boot order of existing devices. Any change or addition of boot devices must be done from the operating system, which is not possible under OpenBSD.
> I booted to a USB key with OpenBSD on it and manually partitioned the new NVME SSD, then rsynced all of the data over from the old drive, but the laptop would not boot to the new NVME drive, instead showing an error message that there was no bootable OS.
> Eventually I figured out that the GPT table that OpenBSD created on the NVMe disk was wrong due to a [one-off bug in the nvme driver](https://github.com/openbsd/src/commit/dc8298f669ea2d7e18c8a8efea509eed200cb989) which was causing the GPT table to be one sector too large, causing the backup GPT table to be written in the wrong location (and other utilities under Linux to write it over the OpenBSD area). I'm guessing the UEFI firmware would fail to read the bad GPT table on the disk that the boot variable pointed to, then declare that disk as missing, and then remove any variables that pointed to that disk.
OpenBSD Support
> The Mi Air's soldered-on Intel 8260 wireless adapter is supported by OpenBSD's iwm driver, including 802.11n support. The Intel sound chip is recognized by the azalia driver.
> The Synaptics touchpad is connected via I2C, but is not yet supported. I am actively hacking on my dwiic driver to make this work and the touchpad will hopefully operate as a Windows Precision Touchpad via imt so I don't have to write an entirely new Synaptics driver.
> Unfortunately since OpenBSD's inteldrm support that is ported from Linux is lagging quite a bit behind, there is no kernel support for Skylake and Kaby Lake video chips. Xorg works at 1920x1080 through efifb so the machine is at least usable, but X is not very fast and there is a noticeable delay when doing certain redrawing operations in xterm. Screen backlight can be adjusted through my OpenBSD port of intel_backlight. Since there is no hardware graphics support, this also means that suspend and resume do not work because nothing is available to re-POST the video after resume. Having to use efifb also makes it impossible to adjust the screen gamma, so for me, I can't use redshift for comfortable night-time hacking.
Flaws
> Especially taking into account the cheap price of the laptop, it's hard to find faults with the design. One minor gripe is that the edges of the case along the bottom are quite sharp, so when carrying the closed laptop, it can feel uncomfortable in one's hands.
> While all of those things could be overlooked, unfortunately there is also a critical flaw in the rollover support in the keyboard/EC on the laptop. When typing certain combinations of keys quickly, such as holding Shift and typing "NULL", one's fingers may actually hold down the Shift, N, and U keys at the same time for a very brief moment before releasing N. Normally the keyboard/EC would recognize U being pressed after N is already down and send an interrupt for the U key. Unfortunately on this laptop, particular combinations of three keys do not interrupt for the third key at all until the second key is lifted, usually causing the third key not to register at all if typed quickly.
I've been able to reproduce this problem in OpenBSD, Linux, and Windows, with the combinations of at least Shift+N+U and Shift+D+F. Holding Shift and typing the two characters in sequence quickly enough will usually fail to register the final character. Trying the combinations without Shift, using Control or Alt instead of Shift, or other character pairs does not trigger the problem.
This might be a problem in the firmware on the Embedded Controller, or a defect in the keyboard circuitry itself. As I mentioned at the beginning, getting technical support for this machine is difficult because it's only sold in China.
Docker on OpenBSD 6.1-current (https://medium.com/@dave_voutila/docker-on-openbsd-6-1-current-c620513b8110)
Dave Voutila writes:
So here’s the thing. I’m normally a macOS user…all my hardware was designed in Cupertino, built in China. But I’m restless and have been toying with trying to switch my daily machine over to a non-macOS system sort of just for fun. I find Linux messy, FreeBSD not as Apple-laptop-friendly as it should be, and Windows a non-starter. Luckily, I found a friend in Puffy. Switching some of my Apple machines over to dual-boot OpenBSD left a gaping hole in my workflow. Luckily, all the hard work the OpenBSD team has done over the last year seems to have plugged it nicely!
OpenBSD’s hypervisor support officially made it into the 6.1 release, but after some experimentation it was rather time consuming and too fragile to get a Linux guest up and running (i.e. basically the per-requisite for Docker). Others had reported some success starting with QEMU and doing lots of tinkering, but after a wasted evening I figured I’d grab the latest OpenBSD snapshot and try what the openbsd-misc list suggested was improved Linux support in active development.
10 (11) Steps to docker are provided
Step 0 — Install the latest OpenBSD 6.1 snapshot (-current)
Step 1 — Configure VMM/VMD
Step 2 — Grab an Alpine Linux ISO
Step 3 — Make a new virtual disk image
Step 4 — Boot Alpine’s ISO
Step 5 — Inhale that fresh Alpine air
Step 6 — Boot Alpine for Reals
Step 7 — Install Docker
Step 8 — Make a User
Step 9 — Ditch the Serial Console
Step 10 — Test out your Docker instance
I haven’t done it yet, but I plan on installing docker-compose via Python’s pip package manager. I prefer defining containers in the compose files.
PostgreSQL + ZFS Best Practices and Standard Procedures (https://people.freebsd.org/~seanc/postgresql/scale15x-2017-postgresql_zfs_best_practices.pdf)
Slides from Sean Chittenden’s talk about PostgreSQL and ZFS at Scale 15x this spring
Slides start with a good overview of Postgres and ZFS, and how to use them together
To start, it walks through the basics of how PostgreSQL interacts with the filesystem (any filesystem)
Then it shows the steps to take a good backup of PostgreSQL, then how to do it even better with ZFS
Then an intro to ZFS, and how Copy-on-Write changes host PostgreSQL interacts with the filesystem
Overview of how ZFS works
ZFS Tuning tips: Compression, Recordsize, atime, when to use mostly ARC vs sharedbuffer, plus pgrepack
Followed by a discussion of the reliability of SSDs, and their Bit Error Rate (BER)
A good SSD has a 4%/year chance of returning the wrong data. A cheap SSD 34%
If you put 20 SSDs in a database server, that means 58% (Good SSDs) to 99.975% (Lowest quality commercially viable SSD) chance of an error per year
Luckily, ZFS can detect and correct these errors
This applies to all storage, not just SSDs, every device fails
More Advice:
Use quotas and reservations to avoid running out of space
Schedule Periodic Scrubs
One dataset per database
Backups: Live demo of rm -rf’ing the database and getting it back
Using clones to test upgrades on real data
Naming Conventions:
Use a short prefix not on the root filesystem (e.g. /db)
Encode the PostgreSQL major version into the dataset name
Give each PostgreSQL cluster its own dataset (e.g. pgdb01)
Optional but recommended: one database per cluster
Optional but recommended: one app per database
Optional but recommended: encode environment into DB name
Optional but recommended: encode environment into DB username
using ZFS Replication
Check out the full detailed PDF and implement a similar setup for your database needs
***
News Roundup
TrueOS Evolving Its "Stable" Release Cycle (https://www.trueos.org/blog/housekeeping-update-infrastructure-trueos-changes/)
TrueOS is reformulating its Stable branch based on feedback from users. The goal is to have a “release” of the stable branch every 6 months, for those who do not want to live on the edge with the rapid updates of the full rolling release
Most of the TrueOS developers work for iX Systems in their Tennessee office. Last month, the Tennessee office was moved to a different location across town. As part of the move, we need to move all our servers. We’re still getting some of the infrastructure sorted before moving the servers, so please bear with us as we continue this process.
As we’ve continued working on TrueOS, we’ve heard a significant portion of the community asking for a more stable “STABLE” release of TrueOS, maybe something akin to an old PC-BSD version release. In order to meet that need, we’re redefining the TrueOS STABLE branch a bit. STABLE releases are now expected to follow a six month schedule, with more testing and lots of polish between releases. This gives users the option to step back a little from the “cutting edge” of development, but still enjoy many of the benefits of the “rolling release” style and the useful elements of FreeBSD Current.
Critical updates like emergency patches and utility bug fixes are still expected to be pushed to STABLE on a case-by-case basis, but again with more testing and polish. This also applies to version updates of the Lumina and SysAdm projects. New, released work from those projects will be tested and added to STABLE outside the 6 month window as well.
The UNSTABLE branch continues to be our experimental “cutting edge” track, and users who want to follow along with our development and help us or FreeBSD test new features are still encouraged to follow the UNSTABLE track by checking that setting in their TrueOS Update Manager.
With boot environments, it will be easy to switch back and forth, so you can have the best of both worlds. Use the latest bleeding edge features, but knowing you can fall back to the stable branch with just a reboot
As TrueOS evolves, it is becoming clearer that one role of the system is to function as a “test platform” for FreeBSD. In order to better serve this role, TrueOS will support both OpenRC and the FreeBSD RC init systems, giving users the choice to use either system. While the full functionality isn’t quite ready for the next STABLE update, it is planned for addition after the last bit of work and testing is complete. Stay tuned for an upcoming blog post with all the details of this change, along with instructions how to switch between RC and OpenRC.
This is the most important change for me. I used TrueOS as an easy way to run the latest version of -CURRENT on my laptop, to use it as a user, but also to do development. When TrueOS deviates from FreeBSD too much, it lessens the power of my expertise, and complicates development and debugging.
Being able to switch back to RC, even if it takes another minute to boot, will bring TrueOS back to being FreeBSD + GUI and more by default, instead of a science project.
We need both of those things, so having the option, while more work for the TrueOS team, I think will be better for the entire community
***
Logical Domains on SunFire T2000 with OpenBSD/sparc64 (http://www.h-i-r.net/2017/05/logical-domains-on-sunfire-t2000-with.html)
A couple of years ago, I picked up a Sun Fire T2000. This is a 2U rack mount server. Mine came with four 146GB SAS drives, a 32-core UltraSPARC T1 CPU and 32GB of RAM.
Sun Microsystems incorporated Logical Domains (LDOMs) on this class of hardware. You don't often need 32 threads and 32GB of RAM in a single server. LDOMs are a kind of virtualization technology that's a bit closer to bare metal than vmm, Hyper-V, VirtualBox or even Xen. It works a bit like Xen, though. You can allocate processor, memory, storage and other resources to virtual servers on-board, with a blend of firmware that supports the hardware allocation, and some software in userland (on the so-called primary or control domain, similar to Xen DomU) to control it.
LDOMs are similar to what IBM calls Logical Partitions (LPARs) on its Mainframe and POWER series computers. My day job from 2006-2010 involved working with both of these virtualization technologies, and I've kind of missed it.
While upgrading OpenBSD to 6.1 on my T2000, I decided to delve into LDOM support under OpenBSD. This was pretty easy to do, but let's walk through it
Resources:
The ldomctl(8) man page (http://man.openbsd.org/OpenBSD-current/man8/sparc64/ldomctl.8)
tedu@'s write-up on Flak (for a different class of server) (http://www.tedunangst.com/flak/post/OpenBSD-on-a-Sun-T5120)
A Google+ post by bmercer@ (https://plus.google.com/101694200911870273983/posts/jWh4rMKVq97)
Once you get comfortable with the fact that there's a little-tiny computer (the ALOM) powered by VXWorks inside that's acting as the management system and console (there's no screen or keyboard/mouse input), Installing OpenBSD on the base server is pretty straightforward. The serial console is an RJ-45 jack, and, yes, the ubiquitous blue-colored serial console cables you find for certain kinds of popular routers will work fine.
OpenBSD installs quite easily, with the same installer you find on amd64 and i386. I chose to install to /dev/sd0, the first SAS drive only, leaving the others unused. It's possible to set them up in a hardware RAID configuration using tools available only under Solaris, or use softraid(4) on OpenBSD, but I didn't do this.
I set up the primary LDOM to use the first ethernet port, em0. I decided I wanted to bridge the logical domains to the second ethernet port. You could also use a bridge and vether interface, with pf and dhcpd to create a NAT environment, similar to how I networked the vmm(4) systems.
Create an LDOM configuration file. You can put this anywhere that's convenient. All of this stuff was in a "vm" subdirectory of my home. I called it ldom.conf:
domain primary {
vcpu 8
memory 8G
}
domain puffy {
vcpu 8
memory 4G
vdisk "/home/axon/vm/ldom1"
vnet
}
Make as many disk images as you want, and make as many additional domain clauses as you wish. Be mindful of system resources. I couldn't actually allocate a full 32GB of RAM across all the LDOMs
I eventually provisioned seven LDOMs (in addition to the primary) on the T2000, each with 3GB of RAM and 4 vcpu cores. If you get creative with use of network interfaces, virtual ethernet, bridges and pf rules, you can run a pretty complex environment on a single chassis, with services that are only exposed to other VMs, a DMZ segment, and the internal LAN.
A nice tutorial, and an interesting look at an alternative platform that was ahead of its time
***
documentation is thoroughly hard (http://www.tedunangst.com/flak/post/documentation-is-thoroughly-hard)
Ted Unangst has a new post this week about documentation:
Documentation is good, so therefore more documentation must be better, right? A few examples where things may have gotten out of control
A fine example is the old OpenBSD install instructions. Once you’ve installed OpenBSD once or twice, the process is quite simple, but you’d never know this based on reading the instructions. Compare the files for 4.8 INSTALL and 5.8 INSTALL. Both begin with a brief intro to the project. Then 4.8 has an enormous list of mirrors, which seems fairly redundant if you’ve already found the install file. Followed by an enormous list of every supported variant of every supported device. Including a table of IO port configurations for ISA devices. Finally, after 1600 lines of introduction we get to the actual installation instructions. (Compared to line 231 for 5.8.) This includes a full page of text about how to install from tape, which nobody ever does. It took some time to recognize that all this documentation was actually an impediment to new users. Attempting to answer every possible question floods the reader with information for questions they were never planning to ask.
Part of the problem is how the information is organized. Theoretically it makes sense to list supported hardware before instructions. After all, you can’t install anything if it’s not supported, right? I’m sure that was considered when the device list was originally inserted above the install instructions. But as a practical matter, consulting a device list is neither the easiest nor fastest way to determine what actually works.
In the FreeBSD docs tree, we have been doing a facelift project, trying to add ‘quick start’ sections to each chapter to let you get to the more important information first. It is also helpful to move data in the forms of lists and tables to appendices or similar, where they can easily be references, but are not blocking your way to the information you are actually hunting for
An example of nerdview signage (http://languagelog.ldc.upenn.edu/nll/?p=29866). “They have in effect provided a sign that will tell you exactly what the question is provided you can already supply the answer.”
That is, the logical minds of technical people often decide to order information in an order that makes sense to them, rather than in the order that will be most useful to the reader
In the end, I think “copy diskimage to USB and follow prompts” is all the instructions one should need, but it’s hard to overcome the unease of actually making the jump. What if somebody is confused or uncertain? Why is this paragraph more redundant than that paragraph? (And if we delete both, are we cutting too much?)
Sometimes we don’t need to delete the information. Just hide it. The instructions to upgrade to 4.8 and upgrade to 5.8 are very similar, with a few differences because every release is a little bit different. The pages look very different, however, because the not at all recommended kernel free procedure, which takes up half the page, has been hidden from view behind some javascript and only expanded on demand. A casual browser will find the page and figure the upgrade process will be easy, as opposed to some long ordeal.
This is important as well, it was my original motivation for working on the FreeBSD Handbook’s ZFS chapter. The very first section of the chapter was the custom kernel configuration required to run ZFS on i386. That scared many users away. I moved that to the very end, and started with why you might want to use ZFS. Much more approachable.
Sometimes it’s just a tiny detail that’s overspecified. The apmd manual used to explain exactly which CPU idle time thresholds were used to adjust frequency. Those parameters, and the algorithm itself, were adjusted occasionally in response to user feedback, but sometimes the man page lagged behind. The numbers are of no use to a user. They’re not adjustable without recompiling. Knowing that the frequency would be reduced at 85% idle vs 90% idle doesn’t really offer much guidance as to whether to enable auto scaling or not. Deleting this detail ensured the man page was always correct and spares the user the cognitive load of trying to solve an unnecessary math problem.
For fun:
For another humorous example, it was recently observed that the deja-dup package provides man page translations for Australia, Canada, and Great Britain. I checked, the pages are in fact not quite identical. Some contain typo fixes that didn’t propagate to other translations. Project idea: attempt to identify which country has the most users, or most fastidious users, by bug fixes to localized man pages.
lldb on BeagleBone Black (https://lists.freebsd.org/pipermail/freebsd-arm/2017-May/016260.html)
I reliably managed to build (lldb + clang/lld) from the svn trunk of LLVM 5.0.0 on my Beaglebone Black running the latest snapshot (May 20th) of FreeBSD 12.0-CURRENT, and the lldb is working very well, and this includes single stepping and ncurses-GUI mode, while single stepping with the latest lldb 4.0.1 from the ports does not work.
In order to reliably build LLVM 5.0.0 (svn), I set up a 1 GB swap partition for the BBB on a NFSv4 share on a FreeBSD fileserver in my network - I put a howto of the procedure on my BLog: https://obsigna.net/?p=659
The prerequesites on the Beaglebone are:
```
pkg install tmux
pkg install cmake
pkg install python
pkg install libxml2
pkg install swig30
pkg install ninja
pkg install subversion
```
On the FreeBSD fileserver:
```
/pathtothe/bbb_share
svn co http://llvm.org/svn/llvm-project/llvm/trunk llvm
cd llvm/tools
svn co http://llvm.org/svn/llvm-project/cfe/trunk clang
svn co http://llvm.org/svn/llvm-project/lld/trunk lld
svn co http://llvm.org/svn/llvm-project/lldb/trunk lldb
```
+ On the Beaglebone Black:
# mount_nfs -o noatime,readahead=4,intr,soft,nfsv4 server:/path_to_the/bbb_share /mnt
# cd /mnt
# mkdir build
# cmake -DLLVM_TARGETS_TO_BUILD="ARM" -DCMAKE_BUILD_TYPE="MinSizeRel" \
-DLLVM_PARALLEL_COMPILE_JOBS="1" -DLLVM_PARALLEL_LINK_JOBS="1" -G Ninja ..
I execute the actual build command from within a tmux session, so I may disconnect during the quite long (40 h) build:
```
tmux new "ninja lldb install"
```
When debugging in GUI mode using the newly build lldb 5.0.0-svn, I see only a minor issue, namely UTF8 strings are not displayed correctly. This happens in the ncurses-GUI only, and this is an ARM issue, since it does not occur on x86 machines. Perhaps this might be related to the signed/unsigned char mismatch between ARM and x86.
Beastie Bits
Triangle BSD Meetup on June 27th (https://www.meetup.com/Triangle-BSD-Users-Group/events/240247251/)
Support for Controller Area Networks (CAN) in NetBSD (http://www.feyrer.de/NetBSD/bx/blosxom.cgi/nb_20170521_0113.html)
Notes from Monday's meeting (http://mailman.uk.freebsd.org/pipermail/ukfreebsd/2017-May/014104.html)
RunBSD - A site about the BSD family of operating systems (http://runbsd.info/)
BSDCam(bridge) 2017 Travel Grant Application Now Open (https://www.freebsdfoundation.org/blog/bsdcam-2017-travel-grant-application-now-open/)
New BSDMag has been released (https://bsdmag.org/download/nearly-online-zpool-switching-two-freebsd-machines/)
***
Feedback/Questions
Philipp - A show about byhve (http://dpaste.com/390F9JN#wrap)
Jake - byhve Support on AMD (http://dpaste.com/0DYG5BD#wrap)
CY - Pledge and Capsicum (http://dpaste.com/1YVBT12#wrap)
CY - OpenSSL relicense Issue (http://dpaste.com/3RSYV23#wrap)
Andy - Laptops (http://dpaste.com/0MM09EX#wrap)
***
| |||
| 07 Mar 2018 | 236: How a cd works | 02:01:49 | |
We’ll cover OpenBSD’s defensive approach to OS security, help you Understanding Syscall Conventions for Different Platforms, Mishandling SMTP Sender Verification, how the cd command works, and the LUA boot loader coming to FreeBSD.
This episode was brought to you by
Headlines
Pledge: OpenBSD’s defensive approach to OS Security (https://medium.com/@_neerajpal/pledge-openbsds-defensive-approach-for-os-security-86629ef779ce)
The meaning of Pledge is same as in the real world, that is, “a solemn promise or undertaking”.
So, in OpenBSD: Calling pledge in a program means to promise that the program will only use certain resources.
How does it make a program more secure?
It limits the operation of a program. Example: You wrote a program named ‘abc’ that only needed the stdio to just print something to stdout.
You added pledge to use only stdio and nothing else.
Then, a malicious user found out that there is a vulnerability in your program which one can exploit and get into shell (or root shell).
Exploiting your program to open a shell (or root shell) will result in the kernel killing the process with SIGABRT (which cannot be caught/ignored) and will generate a log (which you can find with dmesg).
This happens because before executing other codes of your program, the code first pledges not to use anything other than stdio promise/operations. But, opening a shell or root shell will call several other system-calls which are distributed in lots of other promises like “stdio”, “proc”, “exec” etc. They are all forbidden because the program has already promised not to use any promises other than stdio.
Pledge is not a system call filter. So, it is not used to restrict system calls.
For example,
pledge(“read”,NULL) ? wrong syntax of the pledge()
pledge(“stdio inet”,NULL) ? correct syntax of the pledge()
Pledge works on stdio, dns, inet, etc. promises but not directly on system calls like read, write, etc. And, unique functionality of pledge() is that it works on behavioral approach not just like 1:1 approach with the system calls.
On 11 December 2017, Theo de Raadt said:
List: openbsd-tech
Subject: pledge execpromises
From: Theo de Raadt | |||
| 11 Jun 2020 | 354: ZFS safekeeps data | 00:35:07 | |
FreeBSD 11.4-RC 2 available, OpenBSD 6.7 on a PineBook Pro 64, How OpenZFS Keeps Your Data Safe, Bringing FreeBSD to EC2, FreeBSD 2020 Community Survey, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/)
Headlines
FreeBSD 11.4-RC2 Now Available (https://lists.freebsd.org/pipermail/freebsd-stable/2020-May/092320.html)
The second RC build of the 11.4-RELEASE release cycle is now available.
+ 11.4-RELEASE notes (https://www.freebsd.org/releases/11.4R/relnotes.html) (still in progress at the time of recording)
Install OpenBSD 6.7-current on a PineBook Pro 64 (https://xosc.org/pinebookpro.html)
This document is work in progress and I'll update the date above once I change something. If you have something to add, remarks, etc please contact me. Preferably via Mastodon but other means of communication are also fine.
News Roundup
Understanding How OpenZFS Keeps Your Data Safe (https://www.ixsystems.com/blog/openzfs-keeps-your-data-safe/)
Veteran technology writer Jim Salter wrote an excellent guide on the ZFS file system’s features and performance that we absolutely had to share. There’s plenty of information in the article for ZFS newbies and advanced users alike. Be sure to check out the article over at Ars Technica to learn more about ZFS concepts including pools, vdevs, datasets, snapshots, and replication, just to name a few.
Bringing FreeBSD to ec2 (https://www.lastweekinaws.com/podcast/screaming-in-the-cloud/bringing-freebsd-to-ec2-with-colin-percival/)
Colin is the founder of Tarsnap, a secure online backup service which combines the flexibility and scriptability of the standard UNIX "tar" utility with strong encryption, deduplication, and the reliability of Amazon S3 storage. Having started work on Tarsnap in 2006, Colin is among the first generation of users of Amazon Web Services, and has written dozens of articles about his experiences with AWS on his blog.
FreeBSD 2020 Community Survey (https://www.research.net/r/freebsd-2020-community-survey)
The FreeBSD Core Team invites you to complete the 2020 FreeBSD Community Survey. The purpose of this survey is to collect quantitative data from the public in order to help guide the project’s priorities and efforts. This is only the second time a survey has been conducted by the FreeBSD Project and your input is valued.
The survey will remain open for 14 days and will close on June 16th at 17:00 UTC (Tuesday 10am PDT).
Beastie Bits
FreeBSD Project Proposals (https://www.freebsdfoundation.org/blog/submit-your-freebsd-project-proposal)
TJ Hacking (https://www.youtube.com/channel/UCknj_nW8JWcFJOAbgd5_Zgw)
Scotland Open Source podcast (https://twitter.com/ScotlandOSUM/status/1265987126321188864?s=19)
Next FreeBSD Office Hours on June 24, 2020 (https://wiki.freebsd.org/OfficeHours)
***
Feedback/Questions
Tom - Writing for LPIrstudio (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/354/feedback/Tom%20-%20Wriitng%20for%20LPI.md)
Luke - rstudio (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/354/feedback/Luke%20-%20rstudio.md)
Matt - Vlans and Jails (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/354/feedback/Matt%20-%20Vlans%20and%20Jails.md)
Morgan - Can I get some commentary on this issue (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/354/feedback/Morgan%20-%20Can%20I%20get%20some%20commentary%20on%20this%20issue.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 02 Feb 2023 | 492: Feeling for NetBSD | 00:38:43 | |
Writing your own operating system, Continuous Integration and Quality Assurance Update, feeling for the NetBSD community, Testing wanted: execute-only on amd64, GCC uses Modula-2 and Rust, do they work on OpenBSD, Unix is dead; long live Unix, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Part 1: Writing your own operating system (https://o-oconnell.github.io/2023/01/12/p1os.html)
2022 in Review: Continuous Integration and Quality Assurance Update (https://freebsdfoundation.org/blog/2022-in-review-continuous-integration-and-quality-assurance-update/)
News Roundup
I feel for the NetBSD community (https://rubenerd.com/i-feel-for-the-netbsd-community/)
Testing wanted: execute-only on amd64 (https://www.undeadly.org/cgi?action=article;sid=20230115095258)
GCC now includes Modula-2 and Rust. Do they work on OpenBSD? (https://briancallahan.net/blog/)
Unix is dead. Long live Unix! (https://www.theregister.com/2023/01/17/unix_is_dead/)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
• [Kevin - Advent of Computing podcast covers BSD](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/492/feedback/Kevin%20-%20Advent%20of%20Computing%20podcast%20covers%20BSD.md)
• [ilo - thanks](https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/492/feedback/ilo%20-%20thanks.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 02 Mar 2023 | 496: Hacking the CLI | 00:45:05 | |
Automation and Hacking Your FreeBSD CLI, Run your own instant messaging service on FreeBSD, Watch Netflix on FreeBSD, HardenedBSD January 2023 Status Report, How To Set Up SSH Keys With YubiKey as two-factor authentication, OpenSSH fixes double-free memory bug that’s pokable over the network, A late announcement, but better late than never, Next NYC*BUG and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Automation and Hacking Your FreeBSD CLI (https://klarasystems.com/articles/automation-and-hacking-your-freebsd-cli/)
Run your own instant messaging service on FreeBSD (https://xn--gckvb8fzb.com/run-your-own-instant-messaging-service-on-freebsd/)
News Roundup
Watch Netflix on FreeBSD (https://byte--sized-de.translate.goog/linux-unix/netflix-auf-freebsd-schauen/?_x_tr_sl=de&_x_tr_tl=en&_x_tr_hl=en&_x_tr_pto=wapp)
HardenedBSD January 2023 Status Report (https://hardenedbsd.org/article/shawn-webb/2023-01-31/hardenedbsd-january-2023-status-report)
How To Set Up SSH Keys With YubiKey as two-factor authentication (U2F/FIDO2) (https://www.cyberciti.biz/security/how-to-set-up-ssh-keys-with-yubikey-as-two-factor-authentication-u2f-fido2/)
OpenSSH fixes double-free memory bug that’s pokable over the network (https://nakedsecurity.sophos.com/2023/02/03/openssh-fixes-double-free-memory-bug-thats-pokable-over-the-network/)
A late announcement, but better late than never (https://github.com/chettrick/discobsd/releases/tag/DISCOBSD_2_0)
Next NYC*BUG: March? April? Certainly May! (https://lists.nycbug.org:8443/pipermail/talk/2023-February/018550.html)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Daniel - Plan 9 lives (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/496/feedback/Daniel%20-%20Plan%209%20lives.md)
Jason - nvd driver (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/496/feedback/Jason%20-%20nvd%20driver.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 23 Jul 2014 | 47: DES Challenge IV | 01:32:47 | |
Coming up this week on the show! We've got an interview with Dag-Erling Smørgrav, the current security officer of FreeBSD, to discuss what exactly being in such an important position is like. The latest news, answers to your emails and even some LibreSSL drama, on BSD Now - the place to B.. SD.
This episode was brought to you by
Headlines
g2k14 hackathon reports (http://www.openbsd.org/hackathons.html)
Nearly 50 OpenBSD developers gathered in Ljubljana, Slovenia from July 8-14 for a hackathon
Lots of work got done - in just the first two weeks of July, there were over 1000 commits (http://marc.info/?l=openbsd-cvs&r=1&b=201407&w=2) to their CVS tree
Some of the developers wrote in to document what they were up to at the event
Bob Beck (http://undeadly.org/cgi?action=article&sid=20140713220618) planned to work on kernel stuff, but then "LibreSSL happened" and he spent most of his time working on that
Miod Vallat (http://undeadly.org/cgi?action=article&sid=20140718072312) also tells about his LibreSSL experiences
Brent Cook (http://undeadly.org/cgi?action=article&sid=20140718090456), a new developer, worked mainly on the portable version of LibreSSL (and we'll be interviewing him next week!)
Henning Brauer (http://undeadly.org/cgi?action=article&sid=20140714094454) worked on VLAN bpf and various things related to IPv6 and network interfaces (and he still hates IPv6)
Martin Pieuchot (http://undeadly.org/cgi?action=article&sid=20140714191912) fixed some bugs in the USB stack, softraid and misc other things
Marc Espie (http://undeadly.org/cgi?action=article&sid=20140714202157) improved the package code, enabling some speed ups, fixed some ports that broke with LibreSSL and some of the new changes and also did some work on ensuring snapshot consistency
Martin Pelikan (http://undeadly.org/cgi?action=article&sid=20140715120259) integrated read-only ext4 support
Vadim Zhukov (http://undeadly.org/cgi?action=article&sid=20140715094848) did lots of ports work, including working on KDE4
Theo de Raadt (http://undeadly.org/cgi?action=article&sid=20140715212333) created a new, more secure system call, "sendsyslog" and did a lot of work with /etc, sysmerge and the rc scripts
Paul Irofti (http://undeadly.org/cgi?action=article&sid=20140718134017) worked on the USB stack, specifically for the Octeon platform
Sebastian Benoit (http://undeadly.org/cgi?action=article&sid=20140719104939) worked on relayd filters and IPv6 code
Jasper Lievisse Adriaanse (http://undeadly.org/cgi?action=article&sid=20140719134058) did work with puppet, packages and the bootloader
Jonathan Gray (http://undeadly.org/cgi?action=article&sid=20140719082410) imported newer Mesa libraries and did a lot with Xenocara, including work in the installer for autodetection
Stefan Sperling (http://undeadly.org/cgi?action=article&sid=20140721125235) fixed a lot of issues with wireless drivers
Florian Obser (http://undeadly.org/cgi?action=article&sid=20140721125020) did many things related to IPv6
Ingo Schwarze (http://undeadly.org/cgi?action=article&sid=20140721090411) worked on mandoc, as usual, and also rewrote the openbsd.org man.cgi interface
Ken Westerback (http://undeadly.org/cgi?action=article&sid=20140722071413) hacked on dhclient and dhcpd, and also got dump working on 4k sector drives
Matthieu Herrb (http://undeadly.org/cgi?action=article&sid=20140723142224) worked on updating and modernizing parts of xenocara
***
FreeBSD pf discussion takes off (https://lists.freebsd.org/pipermail/freebsd-questions/2014-July/259292.html)
Concerns from last week, about FreeBSD's packet filter being old and unmaintained, seemed to have finally sparked some conversation about the topic on the "questions" and "current" mailing lists (unfortunately people didn't always use reply-all so you have to cross-reference the two lists to follow the whole conversation sometimes)
Straight from the SMP FreeBSD pf maintainer: "no one right now [is actively developing pf on FreeBSD]"
Searching for documentation online for pf is troublesome because there are two incompatible syntaxes
FreeBSD's pf man pages are lacking, and some of FreeBSD's documentation still links to OpenBSD's pages, which won't work anymore - possibly turning away would-be BSD converts because it's frustrating
There's also the issue of importing patches from pfSense, but most of those still haven't been done either
Lots of disagreement among developers vs. users...
Many users are very vocal about wanting it updated, saying the syntax change is no big deal and is worth the benefits - developers aren't interested
Henning Brauer, the main developer of pf on OpenBSD, has been very nice and offered to help the other BSDs get their pf fixed on multiple occasions
Gleb Smirnoff, author of the FreeBSD-specific SMP patches, questions Henning's claims about OpenBSD's improved speed as "uncorroborated claims" (but neither side has provided any public benchmarks)
Gleb had to abandon his work on FreeBSD's pf because funding ran out
***
LibreSSL progress update (http://linux.slashdot.org/story/14/07/16/1950235/libressl-prng-vulnerability-patched)
LibreSSL's first few portable releases have come out and they're making great progress, releasing 2.0.3 two days ago (http://marc.info/?l=openbsd-tech&m=140599450206255&w=2)
Lots of non-OpenBSD people are starting to contribute, sending in patches via the tech mailing list
However, there has already been some drama... with Linux users
There was a problem with Linux's PRNG, and LibreSSL was unforgiving (https://twitter.com/MiodVallat/status/489122763610021888) of it, not making an effort to randomize something that could not provide real entropy
This "problem" doesn't affect OpenBSD's native implementation, only the portable version
The developers (http://www.securityweek.com/openbsd-downplays-prng-vulnerability-libressl) decide to weigh in (http://www.tedunangst.com/flak/post/wrapping-pids-for-fun-and-profit) to calm the misinformation and rage
A fix was added in 2.0.2, and Linux may even get a new system call (http://thread.gmane.org/gmane.linux.kernel.cryptoapi/11666) to handle this properly now - remember to say thanks, guys
Ted Unangst (http://www.bsdnow.tv/episodes/2014_02_05-time_signatures) has a really good post (http://www.tedunangst.com/flak/post/this-is-why-software-sucks) about the whole situation, definitely check it out
As a follow-up from last week, bapt says they're working on building the whole FreeBSD ports tree against LibreSSL, but lots of things still need some patching to work properly - if you're a port maintainer, please test your ports against it
***
Preparation for NetBSD 7 (http://mail-index.netbsd.org/current-users/2014/07/13/msg025234.html)
The release process for NetBSD 7.0 is finally underway
The netbsd-7 CVS branch should be created around July 26th, which marks the start of the first beta period, which will be lasting until September
If you run NetBSD, that'll be a great time to help test on as many platforms as you can (this is especially true on custom embedded applications)
They're also looking for some help updating documentation and fixing any bugs that get reported
Another formal announcement will be made when the beta binaries are up
***
Interview - Dag-Erling Smørgrav - des@freebsd.org (mailto:des@freebsd.org) / @RealEvilDES (https://twitter.com/RealEvilDES)
The role of the FreeBSD Security Officer, recent ports features, various topics
News Roundup
BSDCan ports and packages WG (http://blogs.freebsdish.org/portmgr/2014/07/18/bsdcan-2014-ports-and-packages-wg/)
Back at BSDCan this year, there was a special event for discussion of FreeBSD ports and packages
Bapt talked about package building, poudriere and the systems the foundation funded for compiling packages
There's also some detail about the signing infrastructure and different mirrors
Ports people and source people need to talk more often about ABI breakage
The post also includes information about pkg 1.3, the old pkg tools' EOL, the quarterly stable package sets and a lot more (it's a huge post!)
***
Cross-compiling ports with QEMU and poudriere (http://blog.ignoranthack.me/?p=212)
With recent QEMU features, you can basically chroot into a completely different architecture
This article goes through the process of building ARMv6 packages on a normal X86 box
Note though that this requires 10-STABLE or 11-CURRENT and an extra patch for QEMU right now
The poudriere-devel port now has a "qemu user" option that will pull in all the requirements
Hopefully this will pave the way for official pkgng packages on those lesser-used architectures
***
Cloning FreeBSD with ZFS send (http://blather.michaelwlucas.com/archives/2108)
For a FreeBSD mail server that MWL runs, he wanted to have a way to easily restore the whole system if something were to happen
This post shows his entire process in creating a mirror machine, using ZFS for everything
The "zfs send" and "zfs snapshot" commands really come in handy for this
He does the whole thing from a live CD, pretty impressive
***
FreeBSD Overview series (http://thiagoperrotta.wordpress.com/2014/07/20/here-be-dragons-freebsd-overview-part-i/)
A new blog series we stumbled upon about a Linux user switching to BSD
In part one, he gives a little background on being "done with Linux distros" and documents his initial experience getting and installing FreeBSD 10
He was pleasantly surprised to be able to use ZFS without jumping through hoops and doing custom kernels
Most of what he was used to on Linux was already in the default FreeBSD (except bash...)
Part two (http://thiagoperrotta.wordpress.com/2014/07/21/here-be-packages-freebsd-overview-part-ii/) documents his experiences with pkgng and ports
***
Feedback/Questions
Bostjan writes in (http://slexy.org/view/s214FYbOKL)
Rick writes in (http://slexy.org/view/s21cWLhzj4)
Clint writes in (http://slexy.org/view/s21A4grtH0)
Esteban writes in (http://slexy.org/view/s27fQHz8Se)
Ben writes in (http://slexy.org/view/s21QscO4Cr)
Matt sends in pictures of his FreeBSD CD collection (https://imgur.com/a/Ah444)
***
| |||
| 27 Apr 2023 | 504: Release the BSD | 00:36:06 | |
FreeBSD 13.2 Release, Using DTrace to find block sizes of ZFS, NFS, and iSCSI, Midnight BSD 3.0.1, Closing a stale SSH connection, How to automatically add identity to the SSH authentication agent, Pros and Cons of FreeBSD for virtual Servers, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
FreeBSD 13.2 Release Announcement (https://www.freebsd.org/releases/13.2R/announce/)
Using DTrace to find block sizes of ZFS, NFS, and iSCSI (https://axcient.com/blog/using-dtrace-to-find-block-sizes-of-zfs-nfs-and-iscsi/)
News Roundup
Midnight BSD 3.0.1 (https://www.phoronix.com/news/MidnightBSD-3.0.1)
Closing a stale SSH connection (https://davidisaksson.dev/posts/closing-stale-ssh-connections/)
How to automatically add identity to the SSH authentication agent (https://sleeplessbeastie.eu/2023/04/10/how-to-automatically-add-identity-to-the-ssh-authentication-agent/)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Dan - ZFS question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/504/feedback/Dan%20-%20ZFS%20question.md)
Matt - Thanks (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/504/feedback/Matt%20-%20Thanks.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 11 Jul 2019 | 306: Comparing Hammers | 00:38:21 | |
Am5x86 based retro UNIX build log, setting up services in a FreeNAS Jail, first taste of DragonflyBSD, streaming Netflix on NetBSD, NetBSD on the last G4 Mac mini, Hammer vs Hammer2, and more.
Headlines
Polprog's Am5x86 based retro UNIX build log (https://polprog.net/blog/486/)
I have recently acquired an Am5x86 computer, in a surprisingly good condition. This is an ongoing project, check this page often for updates!
I began by connecting a front panel. The panel came from a different chassis and is slightly too wide, so I had to attach it with a couple of zip-ties. However, that makes it stick out from the PC front at an angle, allowing easy access when the computer sits at the floor - and thats where it is most of the time. It's not that bad, to be honest, and its way easier to access than it would be, if mounted vertically
There is a mains switch on the front panel because the computer uses an older style power supply. Those power supplies instead of relying on a PSON signal, like modern ATX supplies, run a 4 wire cable to a mains switch. The cable carries live and neutral both ways, and the switch keys in or out the power. The system powers on as soon as the switch is enabled.
Originally there was no graphics card in it. Since a PC will not boot with out a GPU, I had to find one. The mainboard only has PCI and ISA slots, and all the GPUs I had were AGP. Fortunately, I bought a PCI GPU hoping it would solve my issue...
However the GPU turned out to be faulty. It took me some time to repair it. I had to repair a broken trace leading to one of the EEPROM pins, and replace a contact in the EEPROM's socket. Then I replaced all the electrolytic capacitors on it, and that fixed it for good.
Having used up only one of the three PCI slots, I populated the remaining pair with two ethernet cards. I still have a bunch of ISA slots available, but I have nothing to install there. Yet.
See the article for the rest of the writeup
Setting up services in a FreeNAS Jail (https://www.ixsystems.com/blog/services-in-freenas-jail/)
This piece demonstrates the setup of a server service in a FreeNAS jail and how to share files with a jail using Apache 2.4 as an example. Jails are powerful, self-contained FreeBSD environments with separate network settings, package management, and access to thousands of FreeBSD application packages. Popular packages such as Apache, NGINX, LigHTTPD, MySQL, and PHP can be found and installed with the pkg search and pkg install commands.
This example shows creating a jail, installing an Apache web server, and setting up a simple web page.
NOTE: Do not directly attach FreeNAS to an external network (WAN). Use port forwarding, proper firewalls and DDoS protections when using FreeNAS for external web sites. This example demonstrates expanding the functionality of FreeNAS in an isolated LAN environment.
News Roundup
First taste of DragonflyBSD (https://nanxiao.me/en/first-taste-of-dragonfly-bsd/)
Last week, I needed to pick a BSD Operating System which supports NUMA to do some testing, so I decided to give Dragonfly BSD a shot. Dragonfly BSDonly can run on X86_64 architecture, which reminds me of Arch Linux, and after some tweaking, I feel Dragonfly BSD may be a “developer-friendly” Operating System, at least for me.
I mainly use Dragonfly BSD as a server, so I don’t care whether GUI is fancy or not. But I have high requirements of developer tools, i.e., compiler and debugger. The default compiler of Dragonfly BSD is gcc 8.3, and I can also install clang 8.0.0 from package. This means I can test state-of-the-art features of compilers, and it is really important for me. gdb‘s version is 7.6.1, a little lag behind, but still OK.
Furthermore, the upgradation of Dragonfly BSD is pretty simple and straightforward. I followed document to upgrade my Operating System to 5.6.0 this morning, just copied and pasted, no single error, booted successfully.
Streaming Netflix on NetBSD (https://www.unitedbsd.com/d/68-streaming-netflix-on-netbsd)
Here's a step-by-step guide that allows streaming Netflix media on NetBSD using a intel-haxm accelerated QEMU vm.
Heads-up! Sound doesn't work, but everything else is fine. Please read the rest of this thread for a solution to this!!
“Sudo Mastery 2nd Edition” cover art reveal (https://mwl.io/archives/4320)
I’m about halfway through the new edition of Sudo Mastery. Assuming nothing terrible happens, should have a complete first draft in four to six weeks. Enough stuff has changed in sudo that I need to carefully double-check every single feature. (I’m also horrified by the painfully obsolete versions of sudo shipped in the latest versions of CentOS and Debian, but people running those operating systems are already accustomed to their creaky obsolescence.)
But the reason for this blog post? I have Eddie Sharam’s glorious cover art. My Patronizers saw it last month, so now the rest of you get a turn.
NetBSD on the last G4 Mac mini (https://tenfourfox.blogspot.com/2019/06/and-now-for-something-completely.html)
I'm a big fan of NetBSD. I've run it since 2000 on a Mac IIci (of course it's still running it) and I ran it for several years on a Power Mac 7300 with a G3 card which was the second incarnation of the Floodgap gopher server. Today I also still run it on a MIPS-based Cobalt RaQ 2 and an HP Jornada 690. I think NetBSD is a better match for smaller or underpowered systems than current-day Linux, and is fairly easy to harden and keep secure even though none of these systems are exposed to the outside world.
Recently I had a need to set up a bridge system that would be fast enough to connect two networks and I happened to have two of the "secret" last-of-the-line 1.5GHz G4 Mac minis sitting on the shelf doing nothing. Yes, they're probably outclassed by later Raspberry Pi models, but I don't have to buy anything and I like putting old hardware to good use.
Hammer vs Hammer2 (https://phoronix.com/scan.php?page=news_item&px=DragonFlyBSD-5.6-HAMMER2-Perf)
With the newly released DragonFlyBSD 5.6 there are improvements to its original HAMMER2 file-system to the extent that it's now selected by its installer as the default file-system choice for new installations. Curious how the performance now compares between HAMMER and HAMMER2, here are some initial benchmarks on an NVMe solid-state drive using DragonFlyBSD 5.6.0.
With a 120GB Toshiba NVMe SSD on an Intel Core i7 8700K system, I ran some benchmarks of DragonFlyBSD 5.6.0 freshly installed with HAMMER2 and then again when returning to the original HAMMER file-system that remains available via its installer. No other changes were made to the setup during testing.
And then for the more synthetic workloads it was just a mix. But overall HAMMER2 was performing well during the initial testing and great to see it continuing to offer noticeable leads in real-world workloads compared to the aging HAMMER file-system. HAMMER2 also offers better clustering, online deduplication, snapshots, compression, encryption, and many other modern file-system features.
Beastie Bits
Unix CLI relational database (https://spin.atomicobject.com/2019/06/16/unix-cli-relational-database/)
The TTY demystified (https://www.linusakesson.net/programming/tty/index.php)
Ranger, a console file manager with VI keybindings (https://ranger.github.io/)
Some Unix Humor (https://www.reddit.com/r/unix/comments/c6o5ze/some_unix_humor/)
OpenBSD -import vulkan-loader for Vulkan API support (https://marc.info/?l=openbsd-ports-cvs&m=156121732625604&w=2)
FreeBSD ZFS without drives (https://savagedlight.me/2019/06/09/freebsd-zfs-without-drives/)
Feedback/Questions
Moritz - ARM Builds (http://dpaste.com/175RRAZ)
Dave - Videos (http://dpaste.com/2DYK85B)
Chris - Raspberry Pi4 (http://dpaste.com/1B16QVN)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
| |||
| 02 Jan 2025 | 592: Wohoo, FreeBSD 14.2 | 01:01:36 | |
ZFS Storage Fault Management, FreeBSD 14.2-RELEASE Announcement, I feel that NAT is inevitable even with IPv6, Spell checking in Vim, OpenBSD Memory Conflict Messages, The Biggest Shell Programs in the World, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
ZFS Storage Fault Management (https://klarasystems.com/articles/zfs-storage-fault-management-linux/?utm_source=BSD%20Now&utm_medium=Podcast)
FreeBSD 14.2-RELEASE Announcement (https://www.freebsd.org/releases/14.2R/announce/)
News Roundup
I feel that NAT is inevitable even with IPv6 (https://utcc.utoronto.ca/~cks/space/blog/tech/IPv6AndStillHavingNAT)
Spell checking in Vim (https://www.tumfatig.net/2024/spell-checking-in-vim/)
OpenBSD Memory Conflict Messages (https://utcc.utoronto.ca/~cks/space/blog/unix/OpenBSDMemoryConflictMessages)
The Biggest Shell Programs in the World (https://github.com/oils-for-unix/oils/wiki/The-Biggest-Shell-Programs-in-the-World)
Beastie Bits
The Connectivity of Things: Network Cultures since 1832 (https://direct.mit.edu/books/oa-monograph/5866/The-Connectivity-of-ThingsNetwork-Cultures-since)
Initial list of 21 EuroBSDcon 2024 videos released (https://www.undeadly.org/cgi?action=article;sid=20241130184249)
-current now has more flexible performance policy (https://www.undeadly.org/cgi?action=article;sid=20241129093132)
OpenBSD 5.1 on Sun Ultra 5 (https://eggflix.foolbazar.eu/w/fa211a4f-6984-4c03-a6d2-b8c329d9459d)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/592/feedback/Phillip%20-%20regressions.md
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 04 May 2016 | 140: Tracing it back to BSD | 01:31:01 | |
This week on BSDNow, Allan is back in down from Europe! We’ll get to hear some of his wrap-up and get caught up on the latest BSD
This episode was brought to you by
Headlines
FreeBSD Quarterly Report (http://www.freebsd.org/news/status/report-2016-01-2016-03.html)
This quarterly status report starts with a rather interesting introduction by Warren Block
ASLR
Porting CEPH to FreeBSD
RCTL I/O Rate Limiting
The Graphics Stack on FreeBSD (Haswell is in, work is progressing on the next update)
CAM I/O Scheduler
NFS Server updates, working around the 16 group limit, and implementing pNFS, allowing NFS to scale beyond a single server
Static Analysis of the FreeBSD Kernel with PVS Studio
PCI-express HotPlug
GitLab Port committed!
WITHFASTDEPEND and other improvements to the FreeBSD build system
Lots of other interesting stuff
***
A Prog By Any Other Name (http://www.tedunangst.com/flak/post/a-prog-by-any-other-name)
Ted Unangst looks at what goes into the name of a program
“Sometimes two similar programs are really the same program with two names. For example, grep and egrep are two commands that perform very similar functions and are therefore implemented as a single program. Running ls -i and observing the inode number of each file will reveal that there is only one file. Calling the program egrep is a shorthand for -E and does the same thing.”
So BSD provides __progname in libc, so a program can tell what its name is
But, what if it has more than one name?
“In fact, every program has three names: its name in the filesystem, the name it has been invoked with, and whatever it believes its own name to be.”
Of course it is not that easy.
“there’s another set of choices for each name, the full path and the basename”
“It’s even possible on some systems for argv[0] to be NULL.”
He then goes on to rename doas (the OpenBSD light replacement for sudo) to banana and discuss what happens
“On that note, another possible bug is to realize that syslog by default uses progname. A user may be able to evade log monitoring by invoking doas with a different name. (Just fixed.)”
Another interesting article from our friend Ted
***
FreeBSD (https://summerofcode.withgoogle.com/organizations/4892834293350400/) and NetBSD (https://summerofcode.withgoogle.com/organizations/6246531984261120/) Google Summer of Code projects have been announced
Some FreeBSD highlights:
Add SCSI passthrough to CTL (share an optical drive via iSCSI)
Add USB target mode driver based on CTL (share a USB device via iSCSI)
API to link created /dev entries to sysctl nodes
Implement Ethernet Ring Protection Switching (ERPS)
HD Audio device model in userspace for bhyve
Some NetBSD highlights:
Implement Ext4fs support in ReadOnly mode
NPF and blacklistd web interface
Port U-Boot so it can be compiled on NetBSD
Split debug symbols for pkgsrc builds
***
libressl - more vague priomises (http://www.tedunangst.com/flak/post/libressl-more-vague-promises)
We haven’t had a Ted U article on the show as of late, however this week we get several! In his next entry “LibreSSL, more vague promises”
He then goes into some detail on what has happened with LibreSSL in the past while, as well as future plans going forward.
“With an eye to the future, what new promises can we make? Some time ago I joked that we only promised to make a better TLS implementation, not a better TLS. Remains true, but fortunately there are people working on that, too. TLS 1.3 support is on the short term watchlist. The good news is we may be ahead of the game, having already removed compression. How much more work can there be?”
“LibreSSL integrated the draft chacha20-poly1305 construction from BoringSSL. The IETF has since standardized a slightly different version because if it were the same it wouldn’t be different. Support for standard variant, and the beginning of deprecation for the existing code, should be landing very shortly. Incidentally, some people got bent out of shape because shipping chacha20 meant exposing non IANA approved numbers to Internet. No promises that won’t happen again.”
***
Interview - Samy Al Bahra - @0xF390 (https://twitter.com/0xF390)
Backtrace
***
News Roundup
systrace(1) is removed for OpenBSD 6.0 (http://marc.info/?l=openbsd-cvs&m=146161167911029&w=2)
OpenBSD has removed systrace, an older mechanism for limiting what syscalls an application can make
It is mostly replaced by the pledge() system
OpenBSD was the first implementation, most others have been unmaintained for some time
The last reported Linux version was for kernel 2.6.1
NetBSD removed systrace in 2007
***
pfSense Video Series: Comprehensive Guide To pfSense 2.3 (https://www.youtube.com/playlist?list=PLE726R7YUJTePGvo0Zga2juUBxxFTH4Bk)
A series of videos (11 so far), about pfSense
Covers Why you would use it, how to pick your hardware, and installation
Then the series covers some networking basics, to make sure you are up to speed before configuring your pfSense
Then a comprehensive tour of the WebUI
Then goes on to cover graphing, backing up and restoring configuration
There are also videos on running DHCP, NTP, and DNS servers
***
DuckDuckGo announces its 2016 FOSS Donations (https://duck.co/blog/post/303/2016-foss-donations-announcement)
The theme is “raising the standard of trust online”
Supported projects include:
OpenBSD Foundation announces DuckDuckGo as a Gold Sponsor (http://undeadly.org/cgi?action=article&sid=20160503085227&mode=expanded)
the Freedom of the Press Foundation for SecureDrop
the Freenet Project
the CrypTech Project
the Tor Project
Fight for the Future for Save Security
Open Source Technology Improvement Fund for VeraCrypt (based on TrueCrypt)
Riseup Labs for LEAP (LEAP Encryption Access Project)
GPGTools for GPGMail
***
Larry the BSD Guy hangs up his hat at FOSS Force (http://fossforce.com/2016/04/bsd-linuxfest-northwest/)
After 15 years, Larry the BSD Guy has decided to hang it up, and walk into the sunset! (Figuratively of course)
After wrapping up coverage of recent LinuxFest NorthWest (Which he didn’t attend), Larry has decided it’s time for a change and is giving up his column over at FOSS Force, as well as stepping away from all things technical.
His last write-up is a good one, and he has some nice plugs for both Dru Lavigne and Michael Dexter of the BSD community.
He will be missed, but we wish him all the luck with the future! He also puts out the plug that FOSS Force will be needing a new columnist in the near future, so if you are interested please let them know!
***
Beastie Bits
If you sponsored “FreeBSD Mastery: Advanced ZFS”, check your mail box (http://blather.michaelwlucas.com/archives/2648)
pkg-1.7.0 is an order of magnitude slower than pkg-1.6.4 (https://marc.info/?l=freebsd-ports&m=146001143408868&w=2) -- Caused by a problem not in pkg
LinuxFest Northwest 2016 Recap (https://www.ixsystems.com/blog/linuxfest-northwest-2016/)
Dru Lavigne's 'Doc like an Egyption' talk from LFNW (https://www.linuxfestnorthwest.org/2016/sessions/doc-egyptian)
Michael Dexters' 'Switching to BSD from Linux' talk from LFNW (https://www.linuxfestnorthwest.org/2016/sessions/devil-details-switching-bsd-linux)
Michael Dexters' 'Secrets to enduring user groups' talk from LFNW (https://www.linuxfestnorthwest.org/2016/sessions/20-year-and-counting-secrets-enduring-user-groups)
January issue of Freebsd Journal online for free (https://www.freebsdfoundation.org/journal/)
Ghost BSD releases 10.3 Alpha1 for testing (http://ghostbsd.org/10.3_alpha1)
EuroBSDcon 2016 - Call for Papers - Dealine: May 8th (https://www.freebsdnews.com/2016/04/15/eurobsdcon-2016-call-for-papers/)
KnoxBUG Initial Meeting (http://www.knoxbug.org/content/knoxbug-maiden-voyage)
Photos, slides, and videos from the Open Source Data Center Conference (https://www.netways.de/en/events_trainings/osdc/archive/osdc2016/)
***
Feedback/Questions
Mohammad - Replication (http://pastebin.com/KDnyWf6Y)
John - Rolling new packages (http://pastebin.com/mAbRwbEF)
Clint - Unicast (http://pastebin.com/BNa6pyir)
Bill - GhostBSD (http://pastebin.com/KDjS2Hxa)
Charles - BSD Videos (http://pastebin.com/ABUUtzWM)
***
| |||
| 01 Feb 2024 | 544: Geeky weather check | 01:07:08 | |
GPL 3: The Controversial Licensing Model and Potential Solutions,
The Geeks way of checking what the outside weather is like, Alpine on a
FreeBSD Jail, DragonFly BSD on a Thinkpad T480s, Dealing with USB Storage
devices on OmniOS, Creating a Time Capsule instance using Samba, FreeBSD, and
ZFS
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
GPL 3: The Controversial Licensing Model and Potential Solutions (https://klarasystems.com/articles/gpl-3-the-controversial-licensing-model-and-potential-solutions/)
The Geeks way of checking what the outside wheather is like (https://blog.netbsd.org/tnf/entry/the_geeks_way_of_checking)
News Roundup
Alpine on a FreeBSD Jail (https://it-notes.dragas.net/2024/01/18/installing-alpine-linux-on-a-freebsd-jail/)
DragonFly BSD on a Thinkpad T480s (https://git.sr.ht/~tomh/dragonflybsd-on-a-laptop/tree/master/item/README.md)
Dealing with USB Storage devices on OmniOS (https://www.tumfatig.net/2024/dealing-with-usb-storage-devices-on-omnios/)
Creating a Time Capsule instance using Samba, FreeBSD, and ZFS (https://dan.langille.org/2024/01/06/creating-a-time-capsule-instance-using-samba-freebsd-and-zfs-2/)
Conferences
FOSDEM (https://fosdem.org/2024/)
AsiaBSDCon (https://2024.asiabsdcon.org/program.html)
BSDCan (https://www.bsdcan.org/2024/papers.php)
EuroBSDcon (https://2024.eurobsdcon.org/)
Southeast Linuxfest (https://southeastlinuxfest.org/2024/01/self-2024-call-for-participation/)
Dont let the name fool you, SELF is BSD friendly and they'd love to have BSD/Unix Talks if you're in the area. JT is staff at SELF, so he can put in a good word for you. ;)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
Join us and other BSD Fans in our BSD Now Telegram channel (https://t.me/bsdnow)
| |||
| 10 Nov 2022 | 480: OpenBSD 7.2 | 00:48:55 | |
OpenBSD 7.2 and FuguIta have been released, Learn the Whys and Hows with the FreeBSD Sec Team, how to get notified about FreeBSD updates, using unbound for ad blocking on OpenBSD, further memory protections on OpenBSD current, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
OpenBSD 7.2 has been released (https://www.openbsd.org/72.html)
FuguIta 7.2 is out as well (https://fuguita.org/index.php?FuguIta%2F7.2=)
***
### Keeping FreeBSD Secure: Learn the Whys and Hows with the FreeBSD Sec Team (https://freebsdfoundation.org/blog/keeping-freebsd-secure-learn-the-whys-and-hows-with-the-freebsd-sec-team/)
News Roundup
Howto: be notified of FreeBSD upgrades, security updates and package updates at login (https://forums.freebsd.org/threads/howto-be-notified-of-freebsd-upgrades-security-updates-and-package-updates-at-login.86660/)
Ads blocking with OpenBSD unbound(8) (https://www.tumfatig.net/2022/ads-blocking-with-openbsd-unbound8/)
Further memory protections committed to -current (http://undeadly.org/cgi?action=article;sid=20221008100649)
Beastie Bits
• [“OpenBSD Mastery: Filesystems” Print/Ebook Bundle Preorder](https://mwl.io/archives/22352)
• [Klara is hiring a FreeBSD Kernel Developer](https://klarasystems.com/careers/freebsd-kernel-developer/)
• [FreeBSD 12.4-BETA1 Now Available](https://lists.freebsd.org/archives/freebsd-stable/2022-October/000920.html)
• [Hunting kernel lock and interrupt latency](https://mail-index.netbsd.org/tech-kern/2022/10/30/msg028499.html)
• [EuroBSDcon 2022 videos available](https://undeadly.org/cgi?action=article;sid=20221027232308)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Charles - BSD Now Bingo (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/480/feedback/Charles%20-%20BSD%20Now%20Bingo.md)
Jake - FreeBSD Security defaults (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/480/feedback/Jake%20-%20FreeBSD%20Security%20defaults.md)
Sam - FreeBSD and SSDs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/480/feedback/Sam%20-%20FreeBSD%20and%20SSDs.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 15 Oct 2020 | 372: Slow SSD scrubs | 00:48:04 | |
Wayland on BSD, My BSD sucks less than yours, Even on SSDs, ongoing activity can slow down ZFS scrubs drastically, OpenBSD on the Desktop, simple shell status bar for OpenBSD and cwm, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow)
Headlines
Wayland on BSD (https://blog.netbsd.org/tnf/entry/wayland_on_netbsd_trials_and)
After I posted about the new default window manager in NetBSD I got a few questions, including "when is NetBSD switching from X11 to Wayland?", Wayland being X11's "new" rival. In this blog post, hopefully I can explain why we aren't yet!
My BSD sucks less than yours (https://www.bsdfrog.org/pub/events/my_bsd_sucks_less_than_yours-full_paper.pdf)
This paper will look at some of the differences between the FreeBSD and OpenBSD operating systems. It is not intended to be solely technical but will also show the different "visions" and design decisions that rule the way things are implemented. It is expected to be a subjective view from two BSD developers and does not pretend to represent these projects in any way.
Video
+ EuroBSDCon 2017 Part 1 (https://www.youtube.com/watch?v=ZhpaKuXKob4)
+ EuroBSDCon 2017 Part 2 (https://www.youtube.com/watch?v=cYp70KWD824)
News Roundup
Even on SSDs, ongoing activity can slow down ZFS scrubs drastically (https://utcc.utoronto.ca/~cks/space/blog/solaris/ZFSSSDActivitySlowsScrubs)
Back in the days of our OmniOS fileservers, which used HDs (spinning rust) across iSCSI, we wound up changing kernel tunables to speed up ZFS scrubs and saw a significant improvement. When we migrated to our current Linux fileservers with SSDs, I didn't bother including these tunables (or the Linux equivalent), because I expected that SSDs were fast enough that it didn't matter. Indeed, our SSD pools generally scrub like lightning.
OpenBSD on the Desktop (Part I) (https://paedubucher.ch/articles/2020-09-05-openbsd-on-the-desktop-part-i.html)
Let's install OpenBSD on a Lenovo Thinkpad X270. I used this computer for my computer science studies. It has both Arch Linux and Windows 10 installed as dual boot. Now that I'm no longer required to run Windows, I can ditch the dual boot and install an operating system of my choice.
A simple shell status bar for OpenBSD and cwm(1) (https://www.tumfatig.net/20200923/a-simple-shell-status-bar-for-cwm/)
These days, I try to use simple and stock software as much as possible on my OpenBSD laptop. I’ve been playing with cwm(1) for weeks and I was missing a status bar. After trying things like Tint2, Polybar etc, I discovered @gonzalo’s termbar. Thanks a lot!
As I love scripting, I decided to build my own.
Beastie Bits
DragonFly v5.8.3 released to address to issues (http://lists.dragonflybsd.org/pipermail/commits/2020-September/769777.html)
OpenSSH 8.4 released (http://www.openssh.com/txt/release-8.4)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Dane - FreeBSD vs Linux in Microservices and Containters (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/372/feedback/Dane%20-%20FreeBSD%20vs%20Linux%20in%20Microservices%20and%20Containters.md)
Mason - questions.md (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/372/feedback/Mason%20-%20questions.md)
Michael - Tmux License.md (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/372/feedback/Michael%20-%20Tmux%20License.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 25 Feb 2021 | 391: i386 tear shedding | 00:38:55 | |
Follow-up about FreeBSD jail advantages, Install Prometheus, Node Exporter and Grafana, Calibrate your touch-screen on OpenBSD, OPNsense 21.1 Marvelous Meerkat Released, NomadBSD 1.4-RC1, Lets all shed a Tear for 386, find mostly doesn't need xargs today on modern Unixes, OpenBSD KDE Status Report, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow)
Headlines
Follow-up about FreeBSD jail advantages (https://rubenerd.com/follow-up-about-freebsd-jail-advantages/)
I’ll admit I ran a lot of justifications together into a single paragraph because I wanted to get to configuring the jails themselves. They’re also, by and large, not specific to FreeBSD’s flavour of containerisation, though I still think it’s easily the most elegant implementation. Sometimes the simplest solution really is the best one.
History of FreeBSD part 4: TCP/IP (https://klarasystems.com/articles/history-of-freebsd-part-4-bsd-and-tcp-ip/)
How TCP/IP evolved and BSDs special contribution to the history of the Internet
***
FreeBSD: Install Prometheus, Node Exporter and Grafana (https://blog.andreev.it/?p=5289)
FreeBSD comes out of the box with three great tools for monitoring. If you need more info about how these tools work, please read the official documentation. I’ll explain the installation only and creating a simple dashboard.
News Roundup
Calibrate your touch-screen on OpenBSD (https://www.tumfatig.net/20210122/calibrate-your-touch-screen-on-openbsd/)
I didn’t expected it but my refurbished T460s came with a touch-screen. It is recognized by default on OpenBSD and not well calibrated as-is. But that’s really simple to solve.
Lets all shed a Tear for 386 (https://lists.freebsd.org/pipermail/freebsd-announce/2021-January/002006.html)
FreeBSD is designating i386 as a Tier 2 architecture starting with FreeBSD 13.0. The Project will continue to provide release images, binary updates, and pre-built packages for the 13.x branch. However, i386-specific issues (including SAs) may not be addressed in 13.x. The i386 platform will remain Tier 1 on FreeBSD 11.x and 12.x.
OPNsense 21.1 Marvelous Meerkat Released (https://opnsense.org/opnsense-21-1-marvelous-meerkat-released/)
For more than 6 years, OPNsense is driving innovation through modularising and hardening the open source firewall, with simple and reliable firmware upgrades, multi-language support, HardenedBSD security, fast adoption of upstream software updates as well as clear and stable 2-Clause BSD licensing.
NomadBSD 1.4-RC1 (https://nomadbsd.org/index.html#1.4-RC1)
We are pleased to present the first release candidate of NomadBSD 1.4.
find mostly doesn't need xargs today on modern Unixes (https://utcc.utoronto.ca/~cks/space/blog/unix/FindWithoutXargsToday)
I've been using Unix for long enough that 'find | xargs' is a reflex. When I started and for a long time afterward, xargs was your only choice for efficiently executing a command over a bunch of find results.
OpenBSD KDE Status Report (https://undeadly.org/cgi?action=article;sid=20210124113220)
OpenBSD has managed to drop KDE3 and KDE4 in the 6.8 -> 6.9 release cycle. That makes me very happy because it was a big piece of work and long discussions. This of course brings questions: Kde Plasma 5 package missing.
After half a year of work, I managed to successfully update the Qt5 stack to the last LTS version 5.15.2. On the whole, the most work was updating QtWebengine. What a monster! With my CPU power at home, I can build it 1-2 times a day which makes testing a little bit annoying and time intensive.
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Karl - Firefox webcam audio solution (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/391/feedback/Karl%20-%20Firefox%20webcam%20audio%20solution.md)
Michal - openzfs (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/391/feedback/Michal%20-%20openzfs.md)
Dave - bufferbloat (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/391/feedback/Dave%20-%20bufferbloat.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 29 Apr 2021 | 400: FreeBSD became 13 | 01:02:43 | |
FreeBSD 13 is here, multi-factor authentication on OpenBSD, KDE on FreeBSD 2021o2, NetBSD GSoC report, a working D compiler on OpenBSD, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow)
Headlines
FreeBSD 13.0 R Annoucement (https://www.freebsd.org/releases/13.0R/announce/)
• OpenZFS 2.0 (almost 2.1) is included in 13.0
• Removed support for previously-deprecated algorithms in geli(8).
• The armv8crypto(4) driver now supports AES-GCM which is used by IPsec and kernel TLS.
Enable multi-factor authentication on OpenBSD (https://dataswamp.org/~solene/2021-02-06-openbsd-2fa.html)
In this article I will explain how to add a bit more security to your OpenBSD system by adding a requirement for user logging into the system, locally or by ssh. I will explain how to setup 2 factor authentication (2FA) using TOTP on OpenBSD
News Roundup
KDE on FreeBSD 2021o2 (https://euroquis.nl/kde/2021/03/26/freebsd2021o2.html)
Gosh, second octant already! Well, let’s take a look at the big things that happened in KDE-on-FreeBSD in these six-and-a-half weeks.
GSoC Reports: Make system(3), popen(3) and popenve(3) use posix_spawn(3) internally (Final report) (http://blog.netbsd.org/tnf/entry/gsoc_reports_make_system_31)
My code can be found at github.com/teknokatze/src in the gsoc2020 branch, at the time of writing some of it is still missing. The test facilities and logs can be found in github.com/teknokatze/gsoc2020. A diff can be found at github which will later be split into several patches before it is sent to QA for merging.
The initial and defined goal of this project was to make system(3) and popen(3) use posixspawn(3) internally, which had been completed in June. For the second part I was given the task to replace fork+exec calls in our standard shell (sh) in one scenario. Similar to the previous goal we determined through implementation if the initial motivation, to get performance improvements, is correct otherwise we collect metrics for why posixspawn() in this case should be avoided. This second part meant in practice that I had to add and change code in the kernel, add a new public libc function, and understand shell internals.
A working D compiler on OpenBSD (https://undeadly.org/cgi?action=article;sid=20210322080633)
Dr. Brian Robert Callahan (bcallah@) blogged about his work in getting D compiler(s) working under OpenBSD.
+ Full Post (https://briancallahan.net/blog/20210320.html)
Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Vasilis - upgrade question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/400/feedback/Vasilis%20-%20upgrade%20question)
Dennis - zfs questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/400/feedback/Dennis%20-%20zfs%20questions)
Daniel Dettlaff - KTLS question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/400/feedback/dmilith%20-%20KTLS)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 14 Jul 2022 | 463: The 1.0 Legend | 00:55:11 | |
Differences between base and ports LLVM in OpenBSD, Netgraph for FreeBSD’s bhyve Networking, Audio on FreeBSD – Quick Guide, FreeBSD’s Legend starts at 1.0, Hacker News running by FreeBSD, TrueNAS 13, and more
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/bsdnow) and the BSDNow Patreon (https://www.patreon.com/bsdnow)
Headlines
Differences between base and ports LLVM in OpenBSD (https://www.cambus.net/differences-between-base-and-ports-llvm-in-openbsd/)
Using Netgraph for FreeBSD’s bhyve Networking (https://klarasystems.com/articles/using-netgraph-for-freebsds-bhyve-networking/?utm_source=bsdweekly)
News Roundup
Audio on FreeBSD – Quick Guide (https://freebsdfoundation.org/freebsd-project/resources/audio-on-freebsd/)
[Legends start at 1.0! – FreeBSD in 1993]
Part 1 (https://eerielinux.wordpress.com/2022/06/18/legends-start-at-1-0-freebsd-in-1993-pt-1/)
Part 2 (https://eerielinux.wordpress.com/2022/06/19/legends-start-at-1-0-freebsd-in-1993-pt-2/)
***
### Hacker News running by FreeBSD. Take that, Linux! (https://news.ycombinator.com/item?id=16076041)
***
### TrueNAS 13 (https://www.theregister.com/2022/05/11/truenas_13_released/)
***
Beastie Bits
Notable OpenBSD news you may have missed, 2022-06-28 edition (http://undeadly.org/cgi?action=article;sid=20220628135253)
rEFInd design for all the BSDs (https://github.com/indgy/refind-bsd-black)
OpenBGPD 7.4 released (https://undeadly.org/cgi?action=article;sid=20220619185920)
Hotfix GhostBSD 22.06.18 ISO is now available (http://ghostbsd.org/22.06.18_iso_is_now_available)
***
###Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
Brad - Jails Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/463/feedback/Brad%20-%20Jails%20Question.md)
Freezr - A few questions (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/463/feedback/Freezr%20-%20A%20few%20questions.md)
A different Brad - Drive question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/463/feedback/A%20different%20Brad%20-%20Drive%20question.md)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 02 Jul 2020 | 357: Study the Code | 00:37:59 | |
OpenBSD 6.7 on PC Engines, NetBSD code study, DRM Update on OpenBSD, Booting FreeBSD on HPE Microserver SATA port, 3 ways to multiboot, and more.
NOTES
This episode of BSDNow is brought to you by Tarsnap (https://www.tarsnap.com/)
Headlines
OpenBSD 6.7 on PC Engines APU4D4 (https://www.tumfatig.net/20200530/openbsd-6-7-on-pc-engines-apu4d4/)
I just got myself a PC Engines APU4D4. I miss an OpenBSD box providing home services. It’s quite simple to install and run OpenBSD on this machine. And you can even update the BIOS from OpenBSD.
NetBSD code study (http://silas.net.br/codereading/netbsd-code.html)
News Roundup
Booting FreeBSD off the HPE MicroServer Gen8 ODD SATA port (https://rubenerd.com/booting-freebsd-off-the-microserver-odd-sata-port/)
My small homelab post generated a ton of questions and comments, most of them specific to running FreeBSD on the HP MicroServer. I’ll try and answer these over the coming week.
Josh Paxton emailed to ask how I got FreeBSD booting on it, given the unconventional booting limitations of the hardware. I thought I wrote about it a few years ago, but maybe it’s on my proverbial draft heap. If you’re impatient, the script is in my lunchbox.
3 ways to multiboot (https://marc.info/?l=openbsd-misc&m=159146428705118&w=2)
multiboot installation of a BSD system with other operating systems
(OSs) on UEFI hardware is not officially supported by any of the
popular
Beastie Bits
pfSense2.4.5-Release-p1 now available (https://www.netgate.com/blog/pfsense-2-4-5-release-p1-now-available.html)
BSDCan 2020 TomSmyth - OpenBSD And OpenBGPD As ISP Controlplane (https://www.youtube.com/watch?v=_eOVlaYWqS8)
OpenBSD DRM Update (https://undeadly.org/cgi?action=article;sid=20200608075708)
***
###Tarsnap
This weeks episode of BSDNow was sponsored by our friends at Tarsnap, the only secure online backup you can trust your data to. Even paranoids need backups.
Feedback/Questions
James - Apple T2 (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/357/feedback/James%20-%20Apple%20T2)
Michael - Jordyns ZFS Question (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/357/feedback/Michael%20-%20Jordyns%20ZFS%20Question)
Note from JT (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/357/feedback/Note%20from%20JT)
Rob - FreeBSD Freindly Registrar (https://github.com/BSDNow/bsdnow.tv/blob/master/episodes/357/feedback/Rob%20-%20FreeBSD%20Freindly%20Registrar)
Send questions, comments, show ideas/topics, or stories you want mentioned on the show to feedback@bsdnow.tv (mailto:feedback@bsdnow.tv)
***
| |||
| 15 Nov 2017 | 220: Opening ZFS in 2017 | 01:54:36 | |
We have a first PS4 kernel exploit, the long awaited OpenZFS devsummit report by Allan, DragonflyBSD 5.0 is out, we show you vmadm to manage jails, and parallel processing with Unix tools.
This episode was brought to you by
Headlines
The First PS4 Kernel Exploit: Adieu (https://fail0verflow.com/blog/2017/ps4-namedobj-exploit/)
The First PS4 Kernel Exploit: Adieu
Plenty of time has passed since we first demonstrated Linux running on the PS4. Now we will step back a bit and explain how we managed to jump from the browser process into the kernel such that ps4-kexec et al. are usable. Over time, ps4 firmware revisions have progressively added many mitigations and in general tried to lock down the system. This post will mainly touch on vulnerabilities and issues which are not present on the latest releases, but should still be useful for people wanting to investigate ps4 security.
Vulnerability Discovery
As previously explained, we were able to get a dump of the ps4 firmware 1.01 kernel via a PCIe man-in-the-middle attack. Like all FreeBSD kernels, this image included “export symbols” - symbols which are required to perform kernel and module initialization processes. However, the ps4 1.01 kernel also included full ELF symbols (obviously an oversight as they have been removed in later firmware versions). This oversight was beneficial to the reverse engineering process, although of course not a true prerequisite. Indeed, we began exploring the kernel by examining built-in metadata in the form of the syscall handler table - focusing on the ps4-specific entries.
Each process object in the kernel contains its own “idt” (ID Table) object. As can be inferred from the snippet above, the hash table essentially just stores pointers to opaque data blobs, along with a given kind and name. Entries may be accessed (and thus “locked”) with either read or write intent. Note that IDTTYPE is not a bitfield consisting of only unique powers of 2. This means that if we can control the kind of an identry, we may be able to cause a type confusion to occur (it is assumed that we may control name).
Exploitation
To an exploiter without ps4 background, it might seem that the easiest way to exploit this bug would be to take advantage of the write off the end of the malloc’d namedobjusrt object. However, this turns out to be impossible (as far as I know) because of a side effect of the ps4 page size being changed to 0x4000 bytes (from the normal of 0x1000). It appears that in order to change the page size globally, the ps4 kernel developers opted to directly change the related macros. One of the many changes resulting from this is that the smallest actual amount of memory which malloc may give back to a caller becomes 0x40 bytes. While this also results in tons of memory being completely wasted, it does serve to nullify certain exploitation techniques (likely completely by accident…).
Adieu
The namedobj exploit was present and exploitable (albeit using a slightly different method than described here) until it was fixed in firmware version 4.06. This vulnerability was also found and exploited by (at least) Chaitin Tech, so props to them! Taking a quick look at the 4.07 kernel, we can see a straightforward fix (4.06 is assumed to be identical - only had 4.07 on hand while writing this post):
int sys_namedobj_create(struct thread *td, void *args) {
// ...
rv = EINVAL;
kind = *((_DWORD *)args + 4)
if ( !(kind & 0x4000) && *(_QWORD *)args ) {
// ... (unchanged)
}
return rv;
}
And so we say goodbye to a nice exploit. I hope you enjoyed this blast from the past :) Keep hacking!
OpenZFS Developer Summit 2017 Recap (https://www.ixsystems.com/blog/openzfs-devsummit-2017/)
The 5th annual OpenZFS Developer Summit was held in San Francisco on October 24-25. Hosted by Delphix at the Children’s Creativity Museum in San Francisco, over a hundred OpenZFS contributors from a wide variety of companies attended and collaborated during the conference and developer summit. iXsystems was a Gold sponsor and several iXsystems employees attended the conference, including the entire Technical Documentation Team, the Director of Engineering, the Senior Analyst, a Tier 3 Support Engineer, and a Tier 2 QA Engineer.
Day 1 of the conference had 9 highly detailed, informative, and interactive technical presentations from companies which use or contribute to OpenZFS. The presentations highlighted improvements to OpenZFS developed “in-house” at each of these companies, with most improvements looking to be made available to the entire OpenZFS community in the near to long term. There’s a lot of exciting stuff happening in the OpenZFS community and this post provides an overview of the presented features and proof-of-concepts.
The keynote was delivered by Mark Maybee who spoke about the past, present, and future of ZFS at Oracle. An original ZFS developer, he outlined the history of closed-source ZFS development after Oracle’s acquisition of Sun. ZFS has a fascinating history, as the project has evolved over the last decade in both open and closed source forms, independent of one another. While Oracle’s proprietary internal version of ZFS has diverged from OpenZFS, it has implemented many of the same features. Mark was very proud of the work his team had accomplished over the years, claiming Oracle’s ZFS products have accounted for over a billion dollars in sales and are used in the vast majority of Fortune 100 companies. However, with Oracle aggressively moving into cloud storage, the future of closed source ZFS is uncertain. Mark presented a few ideas to transform ZFS into a mainstream and standard file system, including adding more robust support for Linux.
Allan Jude from ScaleEngine talked about ZStandard, a new compression method he is developing in collaboration with Facebook. It offers compression comparable to gzip, but at speeds fast enough to keep up with hard drive bandwidth. According to early testing, it improves both the speed and compression efficiency over the current LZ4 compression algorithm. It also offers a new “dictionary” feature for improving image compression, which is of particular interest to Facebook. In addition, when using ZFS send and receive, it will adapt the compression ratio to make the most efficient use of the network bandwidth.
Currently, deleting a clone on ZFS is a time-consuming process, especially when dealing with large datasets that have diverged over time. Sara Hartse from Delphix described how “clone fast delete” speeds up clone deletion. Rather than traversing the entire dataset during clone deletion, changes to the clone are tracked in a “live list” which the delete process uses to determine which blocks to free. In addition, rather than having to wait for the clone to finish, the delete process backgrounds the task so you can keep working without any interruptions. Sara shared the findings of a test they ran on a clone with 500MB of data, which took 45 minutes to delete with the old method, and under a minute using the live list. This behavior is an optional property as it may not be appropriate for long-lived clones where deletion times are not a concern. At this time, it does not support promoted clones.
Olaf Faaland from Lawrence Livermore National Labs demonstrated the progress his team has made to improve ZFS pool imports with MMP (Multi-Modifier Protection), a watchdog system to make sure that ZFS pools in clustered High Availability environments are not imported by more than one host at a time. MMP uses uberblocks and other low-level ZFS features to monitor pool import status and otherwise safeguard the import process. MMP adds fields to on-disk metadata so it does not depend on hardware, such as SAS. It supports multi-node HA configs and does not affect non-HA systems. However, it does have issues with long I/O delays so existing HA software is recommended as an additional fallback.
Jörgen Lundman of GMO Internet gave an entertaining talk on the trials and tribulations of porting ZFS to OS X. As a bonus, he talked about porting ZFS to Windows, and showed a working demo. While not yet in a usable state, it demonstrated a proof-of-concept of ZFS support for other platforms.
Serapheim Dimitropoulos from Delphix discussed Faster Allocation with the Log Spacemap as a means of optimizing ZFS allocation performance. He began with an in-depth overview of metaslabs and how log spacemaps are used to track allocated and freed blocks. Since blocks are only allocated from loaded metaslabs but freed blocks may apply to any metaslab, over time logging the freed blocks to each appropriate metaslab with every txg becomes less efficient. Their solution is to create a pool-wide metaslab for unflushed entries.
Shailendra Tripathi from Tegile presented iFlash: Dynamic Adaptive L2ARC Caching. This was an interesting talk on what is required to allow very different classes of resources to share the same flash device–in their case, ZIL, L2ARC, and metadata. To achieve this, they needed to address the following differences for each class: queue priority, metaslab load policy, allocation, and data protection (as cache has no redundancy).
Isaac Huang of Intel introduced DRAID, or parity declustered RAID. Once available, this will provide the same levels of redundancy as traditional RAIDZ, providing the administrator doubles the amount of options for providing redundancy for their use case. The goals of DRAID are to address slow resilvering times and the write throughput of a single replacement drive being a bottleneck. This solution skips block pointer tree traversal when rebuilding the pool after drive failure, which is the cause of long resilver times. This means that redundancy is restored quickly, mitigating the risk of losing additional drives before the resilver completes, but it does require a scrub afterwards to confirm data integrity. This solution supports logical spares, which must be defined at vdev creation time, which are used to quickly restore the array.
Prakash Surya of Delphix described how ZIL commits currently occur in batches, where waiting threads have to wait for the batch to complete. His proposed solution was to replace batch commits and to instead notify the waiting thread after its ZIL commit in order to greatly increase throughput. A new tunable for the log write block timeout can also be used to log write blocks more efficiently.
Overall, the quality of the presentations at the 2017 OpenZFS conference was high. While quite technical, they clearly explained the scope of the problems being addressed and how the proposed solutions worked. We look forward to seeing the described features integrated into OpenZFS. The videos and slides for the presentations should be made available over the next month or so at the OpenZFS website.
OpenZFS Photo Album (https://photos.google.com/share/AF1QipNxYQuOm5RDxRgRQ4P8BhtoLDpyCuORKWiLPT0WlvUmZYDdrX3334zu5lvY_sxRBA?key=MW5fR05MdUdPaXFKVDliQVJEb3N3Uy1uMVFFdVdR)
DragonflyBSD 5.0 (https://www.dragonflybsd.org/release50/)
DragonFly version 5.0 brings the first bootable release of HAMMER2, DragonFly's next generation file system.
HAMMER2
Preliminary HAMMER2 support has been released into the wild as-of the 5.0 release. This support is considered EXPERIMENTAL and should generally not yet be used for production machines and important data. The boot loader will support both UFS and HAMMER2 /boot. The installer will still use a UFS /boot even for a HAMMER2 installation because the /boot partition is typically very small and HAMMER2, like HAMMER1, does not instantly free space when files are deleted or replaced.
DragonFly 5.0 has single-image HAMMER2 support, with live dedup (for cp's), compression, fast recovery, snapshot, and boot support. HAMMER2 does not yet support multi-volume or clustering, though commands for it exist. Please use non-clustered single images for now.
ipfw Updates
IPFW has gone through a number of updates in DragonFly and now offers better performance. pf and ipfw3 are also still supported.
Improved graphics support
The i915 driver has been brought up to match what's in the Linux 4.7.10 kernel. Intel GPUs are supported up to the Kabylake generation.
vga_switcheroo(4) module added, allowing the use of Intel GPUs on hybrid-graphics systems.
The new apple_gmux driver enables switching to the Intel video chipset on dual Intel/NVIDIA and Intel/Radeon Macbook computers.
Other user-affecting changes
efisetup(8) added.
DragonFly can now support over 900,000 processes on a single machine.
Client-side SSH by default does not try password authentication, which is the default behavior in newer versions of OpenSSH. Pass an explicit '-o PasswordAuthentication=yes' or change /etc/ssh/ssh_config if you need the old behavior. Public key users are unaffected.
Clang status
A starting framework has been added for using clang as the alternate base compiler in DragonFly, to replace gcc 4.7. It's not yet complete. Clang can of course be added as a package.
Package updates
Many package updates but I think most notably we need to point to chrome60 finally getting into dports with accelerated video and graphics support.
64-bit status
Note that DragonFly is a 64-bit-only operating system as of 4.6, and will not run on 32-bit hardware.
AMD Ryzen is supported and DragonFly 5.0 has a workaround for a hardware bug (http://lists.dragonflybsd.org/pipermail/commits/2017-August/626190.html).
DragonFly quickly released a v5.0.1 with a few patches Download link (https://www.dragonflybsd.org/download/)
News Roundup
(r)vmadm – managing FreeBSD jails (https://blog.project-fifo.net/rvmadm-managing-freebsd-jails/)
We are releasing the first version (0.1.0) of our clone of vmadm for FreeBSD jails today. It is not done or feature complete, but it does provides basic functionality. At this point, we think it would be helpful to get it out there and get some feedback. As of today, it allows basic management of datasets, as well as creating, starting, stopping, and destroying jails.
Why another tool to manage jails
However, before we go into details let’s talk why we build yet another jail manager? It is not the frequent NIH syndrome, actually quite the opposite. In FiFo 0.9.2 we experimented with iocage as a way to control jails. While iocage is a useful tool when used as a CLI utility it has some issues when used programmatically.
When managing jails automatically and not via a CLI tool things like performance, or a machine parsable interface matter. While on a CLI it is acceptable if a call takes a second or two, for automatically consuming a tool this delay is problematic.
Another reason for the decision was that vmadm is an excellent tool. It is very well designed. SmartOs uses vmadm for years now. Given all that, we opted for adopting a proven interface rather than trying to create a new one. Since we already interface with it on SmartOS, we can reuse a majority of our management code between SmartOS and FreeBSD.
What can we do
Today we can manage datasets, which are jail templates in the form of ZFS volumes. We can list and serve them from a dataset-server, and fetch those we like want. At this point, we provide datasets for FreeBSD 10.0 to 11.1, but it is very likely that the list will grow. As an idea here is a community-driven list of datasets (https://datasets.at/) that exist for SmartOS today. Moreover, while those datasets will not work, we hope to see the same for BSD jails.
After fetching the dataset, we can define jails by using a JSON file. This file is compatible with the zone description used on SmartOS. It does not provide all the same features but a subset. Resources such as CPU and memory can be defined, networking configured, a dataset selected and necessary settings like hostname set.
With the jail created, vmadm allows managing its lifetime, starting, stopping it, accessing the console and finally destroying it. Updates to jails are supported to however as of today they are only taken into account after restarting the jail. However, this is in large parts not a technical impossibility but rather wasn’t high up on the TODO list.
It is worth mentioning that vmadm will not pick up jails created in other tools or manually. Only using vmadm created jails was a conscious decision to prevent it interfering with existing setups or other utilities. While conventional tools can manage jails set up with vmadm just fine we use some special tricks like nested jails to allow for restrictions required for multi-tenancy that are hard or impossible to achieve otherwise.
Whats next
First and foremost we hope to get some feedback and perhaps community engagement. In the meantime, as announced earlier this year (https://blog.project-fifo.net/fifo-in-2017/), we are hard at work integrating FreeBSD hypervisors in FiFo, and as of writing this, the core actions work quite well.
Right now only the barebone functions are supported, some of the output is not as clear as we would like. We hope to eventually add support for behyve to vmadm the same way that it supports KVM on SmartOS. Moreover, the groundwork for this already exists in the nested jail techniques we are using.
Other than that we are exploring ways to allow for PCI pass through in jails, something not possible in SmartOS zones right now that would be beneficial for some users.
In general, we want to improve compatibility with SmartOS as much as possible and features that we add over time should make the specifications invalid for SmartOS.
You can get the tool from github (https://github.com/project-fifo/r-vmadm).
***
Parallel processing with unix tools (http://www.pixelbeat.org/docs/unix-parallel-tools.html)
There are various ways to use parallel processing in UNIX:
piping
An often under appreciated idea in the unix pipe model is that the components of the pipe run in parallel. This is a key advantage leveraged when combining simple commands that do "one thing well"
split -n, xargs -P, parallel
Note programs that are invoked in parallel by these, need to output atomically for each item processed, which the GNU coreutils are careful to do for factor and sha*sum, etc. Generally commands that use stdio for output can be wrapped with the stdbuf -oL command to avoid intermixing lines from parallel invocations
make -j
Most implementations of make(1) now support the -j option to process targets in parallel. make(1) is generally a higher level tool designed to process disparate tasks and avoid reprocessing already generated targets. For example it is used very effictively when testing coreutils where about 700 tests can be processed in 13 seconds on a 40 core machine.
implicit threading
This goes against the unix model somewhat and definitely adds internal complexity to those tools. The advantages can be less data copying overhead, and simpler usage, though its use needs to be carefully considered. A disadvantage is that one loses the ability to easily distribute commands to separate systems. Examples are GNU sort(1) and turbo-linecount
The example provided counts lines in parallel:
The examples below will compare the above methods for implementing multi-processing, for the function of counting lines in a file. First of all let's generate some test data. We use both long and short lines to compare the overhead of the various methods compared to the core cost of the function being performed:
$ seq 100000000 > lines.txt # 100M lines
$ yes $(yes longline | head -n9) | head -n10000000 > long-lines.txt # 10M lines
We'll also define the add() { paste -d+ -s | bc; } helper function to add a list of numbers.
Note the following runs were done against cached files, and thus not I/O bound. Therefore we limit the number of processes in parallel to $(nproc), though you would generally benefit to raising that if your jobs are waiting on network or disk etc.
+ We'll use this command to count lines for most methods, so here is the base non multi-processing performance for comparison:
$ time wc -l lines.txt
$ time wc -l long-lines.txt
split -n
Note using -n alone is not enough to parallelize. For example this will run serially with each chunk, because since --filter may write files, the -n pertains to the number of files to split into rather than the number to process in parallel.
$ time split -n$(nproc) --filter='wc -l' lines.txt | add
You can either run multiple invocations of split in parallel on separate portions of the file like:
$ time for i in $(seq $(nproc)); do
split -n$i/$(nproc) lines.txt | wc -l&
done | add
Or split can do parallel mode using round robin on each line, but that's huge overhead in this case. (Note also the -u option significant with -nr):
$ time split -nr/$(nproc) --filter='wc -l' lines.txt | add
Round robin would only be useful when the processing per item is significant.
Parallel isn't well suited to processing a large single file, rather focusing on distributing multiple files to commands. It can't efficiently split to lightweight processing if reading sequentially from pipe:
$ time parallel --will-cite --block=200M --pipe 'wc -l' < lines.txt | add
Like parallel, xargs is designed to distribute separate files to commands, and with the -P option can do so in parallel. If you have a large file then it may be beneficial to presplit it, which could also help with I/O bottlenecks if the pieces were placed on separate devices:
split -d -n l/$(nproc) lines.txt l.
Those pieces can then be processed in parallel like:
$ time find -maxdepth 1 -name 'l.*' |
xargs -P$(nproc) -n1 wc -l | cut -f1 -d' ' | add
If your file sizes are unrelated to the number of processors then you will probably want to adjust -n1 to batch together more files to reduce the number of processes run in total. Note you should always specify -n with -P to avoid xargs accumulating too many input items, thus impacting the parallelism of the processes it runs.
make(1) is generally used to process disparate tasks, though can be leveraged to provide low level parallel processing on a bunch of files. Note also the make -O option which avoids the need for commands to output their data atomically, letting make do the synchronization. We'll process the presplit files as generated for the xargs example above, and to support that we'll use the following Makefile:
%: FORCE # Always run the command
@wc -l < $@
FORCE: ;
Makefile: ; # Don't include Makefile itself
One could generate this and pass to make(1) with the -f option, though we'll keep it as a separate Makefile here for simplicity. This performs very well and matches the performance of xargs.
$ time find -name 'l.*' -exec make -j$(nproc) {} + | add
Note we use the POSIX specified "find ... -exec ... {} +" construct, rather than conflating the example with xargs. This construct like xargs will pass as many files to make as possible, which make(1) will then process in parallel.
OpenBSD gives a hint on forgetting unlock mutex (http://nanxiao.me/en/openbsd-gives-a-hint-on-forgetting-unlock-mutex/)
OpenBSD gives a hint on forgetting unlock mutex
Check following simple C++ program:
> ```
#include | |||
Enhance your understanding of BSD Now with My Podcast Data
At My Podcast Data, we strive to provide in-depth, data-driven insights into the world of podcasts. Whether you're an avid listener, a podcast creator, or a researcher, the detailed statistics and analyses we offer can help you better understand the performance and trends of BSD Now. From episode frequency and shared links to RSS feed health, our goal is to empower you with the knowledge you need to stay informed and make the most of your podcasting experience. Explore more shows and discover the data that drives the podcast industry.
© My Podcast Data