Security Team Post-FAD Notes
On 11 March, some of the Security Team met in Washington, D.C. for a day-long FAD where we discussed several issues. Zach Oglesby released his notes from the meeting and I’ll be using those to describe my take-away from the meeting.
Right now Fedora has a couple of problems with getting security fixes onto people’s systems quickly. The first has to do with embargoes. Because Fedora isn’t part of the trusted network, we don’t get advance notification of vulnerabilities before their embargo is lifted. This means that when we are notified of an urgent security vulnerability the public also has the information and we’re left scrambling to find the fix (patch, new version, etc) and ship it. Some other flavors of Linux will have had advance notice and will have these patches or new versions packaged up and ready to ship with the embargo is lifted.
The other problem deals with the Fedora Mirror network. Because there are many mirrors it could take many hours or days(?) to get a security update out to all Fedora users. This has to do with the mechanisms involved for keeping the Mirror system efficient; not necessarily fast. We discussed potential solutions for this problem as well.
Working with embargoes
Our solution to working with embargoes is to create a trusted team, with the appropriate tools, to deal with these issues so we can get a head start on urgent security vulnerabilities.
Right now the infrastructure isn’t in place to be able to handle embargoed vulnerabilities. Red Hat’s Bugzilla instance is currently designed to hold embargoed information, which is good and we definitely want to leverage that resource, but we need our build system and perhaps even Bodhi to be able to support private builds. The idea is to have the packages built and ready to ship before the embargo expires. When the embargo is lifted we would just need to push the big red button and the packages would ship.
It would also be hoped that the fix would have gone through QA before the end of the embargoed period so that we’re fairly sure that we’re not breaking anything (other than the vulnerability) in the process. For this part, we’re going to need trusted individuals to work on these issues.
Oh, and make no mistake, we aren’t trying to hide information forever. To maintain transparency that Fedora has been build on all of our tools we use to handle embargoed information should be able to make that information public at the end of the embargo. Bugzilla tickets would be opened up to the public and builds should be made available as well.
Trust is a difficult thing to define. How do you establish trust and how do you penalize a break in that trust? Simply put, we need a way to do both before we start handling sensitive information. We might be able to show that we have the systems and the procedures for handling this information but the first time there’s a leak and it comes from inside Fedora it’s quite likely we will lose access to this information and be back to where we are today.
We will be working with legal and Fedora management (FPL, FESCo, and Release Engineering) to devise a plan and determine the best way to involve package maintainers, proven packagers, and QA.
Faster access to Security Fixes
Waiting hours or days for urgent security fixes to become available on mirrors really isn’t acceptable. Recent critical vulnerabilities have seen exploits in the wild shortly after the vulnerability being made public. We need to be able to get fixes out to users faster.
Debian uses separate servers to deliver security fixes until those fixes have propagated out to their normal mirrors. This could work, especially with diff packages being used which are usually much smaller than the full package, but would increase the infrastructure needed to support disseminating packages.
There may also be a way to tweak our existing infrastructure to improve the time it takes to push these urgent packages out. Either way, the Security Team will be working with Release Engineering to help figure out a solution.
Another big topic we covered is training. Many people turn to the Security Team to learn. From a team point of view, we want to make sure our members have a common base of knowledge from which to work. We decided to launch Apprentice and Journey level certifications to will provide this base of knowledge.
We worked through what we would want to see in a new member and created the Security Team Apprenticeship. While not fully complete, we hope to have it ready in the coming months.
We also talked about mentors and what it means to be a mentor.
We didn’t put down many hard and fast rules and that’s okay. We want people to be active and participate. Hopefully the mentors won’t have a heavy lift and neither will the mentees.
Obligatory GPG Key Signing
Of course, a security FAD can’t end without the obligatory GPG key signing. This was completed with two new members of the team.
So what I’ve documented represents a day of work. Thanks to all that participated, even if you only had a chance to pop in, virtually, and lend your opinions or support. Hopefully we can do this again in a few months and work on new tasks.