Hacker Newsnew | past | comments | ask | show | jobs | submitlogin
The pfSense Book (netgate.com)
121 points by ingve on Aug 8, 2018 | hide | past | favorite | 43 comments


A great resource for an excellently flexible and rock-solid reliable platform.

Have been running pfsense and pfsense clusters for many, many years and had practically nothing but ultra-reliable success stories.


> rock-solid reliable platform

Eh... I've had a lot of issues with it on my flakey Charter internet[0] connection. Php-fpm hangs so often I added a cron job to restart it every fifteen minutes, it doesn't always run dhclient when an interface has been down for a bit and then comes back up (cron to the rescue again), the gateway monitor did weird and stupid things and had to be disabled, services aren't always restarted when they crash (Unbound recently) stuff like that.

[0] I am aware that this combination of words is redundant.


Sounds like you had some unrelated hardware issues TBH. I always run either decent quality fanless i3 or quad core celerons (yes I know, friends don’t let friends buy celerons but some recent models have been good bang for buck) for low impact deployments and always a cpu that supports and deployed with ECC memory for high impact / risk deployments because... ECC matters.

Regardless, easy to blame a routing OS for commonly poor quality hardware.


What hardware are you running pfsense on?


Some overpowered former thin client. It isn't likely to be the hardware, which can run Windows 7 comfortably.


‘Windows 7’... doesn’t mean much to be honest, if that OS didn’t corrupt its filesystem it’d crap out on its poor kernel or registry design and regardless why would you run a presumably important routing system on the same hardware you thought was suitable for ‘Windows 7’ (rhetorical).


High end routers made for enterprises run on significantly fewer resources than Windows 7 requires. If pfSense can't manage on this hardware then it is exceptionally bloated for its job.


in a previous life, I had 5 Community version pfSense firewalls running off of Soekris net5505 boxes. They were in areas with a lot of power issues (UPS's helped as much as they can), so they were beaten up pretty bad for 6+ years before I left.

Only issues I had were once I got 50mb up/down with 60+ users, it started to bog down. I virtualized that one in VMWare and that took care of that. I did have one power supply burn out and one CF card die, but those net5501 were in use for 8 years straight before I repurposed them.

For $250 (in 2005 maybe?) for hardware, I can't complain at all.

*Edit

The best part was exporting the backup configs, then tossing them onto a new machine (another Soekris, VM, or PC hardware) and having a new firewall back up in minutes if you had the ethernet ports figured out beforehand.


Similar experiences +/- scale, crazy realibility while immediately applying security / important updates when available.


Anyone had experience using this to setup up an ipsec VPN? I am looking for something that can do ipsec at 10Gbps which seems like it should be possible with an x86 processor that supports AES-NI but a lot of anecdotal benchmarks I've seen on randon websites seem to suggest it is difficult to achieve.


YMMV with Pfsense. I've been using it since Scott and Chris were in Kentucky working on it together... that'll date me for sure...

Anyway, I found that unless someone else has already done it and proven the hardware can handle it then it's all an experiment you conduct on your own dime.

FYI - Netgate, the parent company of PFSense, went after OPNSense by scooping their domain and used it to spread FUD. Just dirty stuff.

https://en.wikipedia.org/wiki/OPNsense


> YMMV with Pfsense. I've been using it since Scott and Chris were in Kentucky working on it together... that'll date me for sure...

You've been "using" it. We've been directly supporting it (and before it, m0n0wall) with cash on the barrel every month since before it was first released (Oct 2006). In Sep 2012, we bought out Scott, and started investing even more heavily in pfSense development.


I also “paid” for netgate hardware and recommended to many companies that they buy it. So in essence I helped you invest in the pfsense development.

What you did was still dirty.


Last summer I was looking into a building a little project that could do just that.

TL, DR: It looks like those with the patience to compile a kernel+drivers can probably manage using Denverton boards [1]. I was blocked by no available drivers last year but hope to come back to my build project in the fall.

NB: I'm just a casual tinkerer not an expert.

Back story: Last summer when Intel shipped Denverton, QAT (which offloads crypto) apparently got a substantial speed boost and I saw claims that 10GbE IPSec should be possible. Denverton boards vary but some top end ones apparently ship with 4 10GbE NICs [2].

Back then the issue was that the QAT drivers weren't ready, so I put my project on hold. Today I see Intel has apparently shipped Linux and FreeBSD drivers. [3] Would of course be nice to see these drivers land upstream but no idea what the licensing story is like here.

[1] https://www.servethehome.com/intel-atom-c3000-series-launch-...

[2] https://www.servethehome.com/supermicro-a2sdi-h-tp4f-review/

[2] https://01.org/intel-quick-assist-technology/downloads


Depending on the SoC, C3000s have 0, 2 or 4 on-die MACs that will do 10Gbps. Also depending on the Doc, C3000s have no QAT or one that will run at "low", "medium" or "high" speed. The difference is in the number of "engines" that can be in-use.

https://ark.intel.com/compare/97935,97928,97930,97929,97937,...

Support for the C3000 NICs is included in pfSense 2.4.4. Support for the C3000 QAT will come later.

I can say that we've tested to > 800Gbps on a C3550 using AES-GCM-128 between a pair of the C3558-based systems we sell. This is using AES-NI, not QAT.


I don't know about FreeBSD, but on Linux WireGuard yields some 15-20% better performance (on both throughput and latency). OpenVPN is completely crap. And both OpenVPN and IPsec are much harder to set up than WireGuard.


> on Linux WireGuard yields some 15-20% better performance (on both throughput and latency).

Assuming you're comparing to IPsec (and AES-NI accelerated AES-GCM on the P2s), and especially a routed IPsec (VTI) .vs the more normally found "policy-based" IPsec, then this isn't true, for at least 'throughput', and it's completely provable.

I've not tested latency.

Agreed with your statements on OpenVPN, and the advantages of setup for WireGuard .vs IPsec or OpenVPN.


If you have the traffic (many streams) it should be possible, yes.

First thing you can do (if you haven't yet) is move from AES-CBC + HMAC-SHA1 to AES-GCM on the P2s, otherwise you're limited to the throughput of SHA1 (AES-NI accelerates AES, but the SHA1 will hold you back. AES-GCM doesn't need the HMAC (it's an AEAD algorithm), so it's faster.

In theory, with AES-NI and AES-GCM, you should be able to get 2gbps per core. In fact, we've proven with with a different packet processing architecture (which I'll get to below).

Problem: the GCM implementation we did for FreeBSD is slower than that, because I took the decision that the ability to resist side-channel attacks was more important that ultimate speed. It's also limited to a single core. There is some recent work in pfSense to use > 1 core for the AES encryption.

So the second thing you can do is set net.inet.ipsec.async_crypto=1 Details are in https://reviews.freebsd.org/D10680 We've exposed a knob for this in pfSense 2.4.4 https://redmine.pfsense.org/issues/8772#change-37604

But after that, you're going to be out of gas. Kernel networking will only go so fast, and the FreeBSD IPsec stack / cryptdev and AES-GCM implementation are all straight-forward packet-at-a-time. The FreeBSD (and linux) kernel stack(s) are inherently packet-at-a-time. This tends to limit available throughput. We are adding support for QAT to pfSense in 2.4.4, so there should be some additional gain there, as well.

That all said, it is possible to get to 10Gbps IPsec. We did some testing on a pair of i7-6950X boxes equipped with 40Gbps NICs. (Details cam be found in https://www.netgate.com/blog/building-a-behemoth-router.html) and achieved the following results. Remember these are on linux and, where noted, VPP. Measurements were to/from two more Xeon boxes equipped with 40gbps NICs as well. Lite details in the slides / video linked below.

- kernel, AES-NI, AES-CBC-128 + HMAC-SHA1, 1 SA, 1 stream: 2.09 Gbps.

- kernel, AES-NI, AES-CBC-128 + HMAC-SHA1, 1 SA, 4 streams: 2.07 Gbps.

- kernel, AES-NI, AES-CBC-128 + HMAC-SHA1, 8 SAs, 8 streams: 10.85 Gbps.

- kernel, AES-NI, AES-GCM-128-16, 1 SA, 1 stream: 5.06 Gbps.

- kernel, AES-NI, AES-GCM-128-16, 1 SA, 4 streams: 5.06 Gbps.

- kernel, AES-NI, AES-GCM-128-16, 8 SAs, 8 streams: 25.25 Gbps.

Note in both the above that using > 1 SA and > 1 'stream' (TCP flow) allows the traffic to be spread across the cores. Remember also that this is linux. FreeBSD isn't able to turn in this type of result.

Then we tested with a CPIC (https://www.netgate.com/products/cpic-8955.html) QuickAssist card in the tunnel endpoints:

- kernel, QAT, AES-CBC-128 + HMAC-SHA1, 1 SA, 1 stream: 8.74 Gbps.

- kernel, QAT, AES-CBC-128 + HMAC-SHA1, 1 SA, 4 streams: 8.74 Gbps.

- kernel, QAT, AES-CBC-128 + HMAC-SHA1, 8 SAs, 8 streams: 27.08 Gbps.

Note that 8.74 Gbps is quite close the the maximum you'll get through a 10 Gbps NIC, once the framing and packet overheads are accounted for.

But look what happens when we move the processing to userspace with VPP:

- vpp, AES-NI, AES-CBC-128 + HMAC-SHA1, 1 SA, 1 stream: 7.42 Gbps.

- vpp, AES-NI, AES-CBC-128 + HMAC-SHA1, 1 SA, 4 streams: 8.28 Gbps.

- vpp, AES-NI, AES-GCM-128-16, 1 SA, 1 stream: 13.70 Gbps.

- vpp, AES-NI, AES-GCM-128-16, 1 SA, 4 streams: 15.93 Gbps.

And with hardware offload, we can get to the maximum possible throughput on a 40Gbps NIC:

- vpp, QAT, AES-CBC-128 + HMAC-SHA1, 1 SA, 1 stream: 32.68 Gbps.

- vpp, QAT, AES-CBC-128 + HMAC-SHA1, 1 SA, 4 streams: 35.72 Gbps.

- vpp, QAT, AES-CBC-128 + HMAC-SHA1, 8 SAs, 8 streams: 36.32 Gbps.

- vpp, QAT, AES-GCM-128-16, 1 SA, 1 stream: 32.73 Gbps.

- vpp, QAT, AES-GCM-128-16, 1 SA, 4 stream: 32.98 Gbps.

Note that this was all with an internal build of VPP (as used in our TNSR product) in April 2017. Things have progressed since then. I talked about this some in 2017.

Video https://www.safaribooksonline.com/library/view/oscon-2017-/9...

Slides https://wiki.fd.io/view/File:40_Gbps_IPsec_on_commodity_hard...

https://www.netgate.com/products/tnsr/


They have a new product called TNSR that looks like it can do this.


and more. See elsewhere in-thread for results.


Use it in prod. 10gbps throughput on egress NAT on commodity x86 hw. 5 stars would recommend.


Anyone used the commercial firewall / IP list you can slap on top of pfSense?

There is the community version, but then there is also this: https://www.proofpoint.com/us/threat-insight/et-pro-ruleset

My understanding is it’s around $900 or so a year for a license. That kind of sucks. But on the other hand I know it’s very easy to install, just a one-liner in the GUI of pfSense (which is how I found out about it).


Community version is 30day old or behind the paid version. Problem with alot of IDS/IPS is it needs to see the contents of the packets before it can block, unless it breaks the encryption its not going to work. So its not a panacea but then nothing is, and its hard to tell if this is already provided by AV products with web browser addons. If you take Kaspersky, they are the only AV product which uses CPU virtualisation to scan for malware running in memory on a windows box for their Safe Money web browser facility. CPU virtualisation is one of those areas that can bypass alot of security products, so if you can stack up enough zero days to get into an OS and mod a bios, you can hide for a long time. Single core CPU's also have their uses as well, but you dont see that mentioned much either in the security arena.


Well Worth It!


PFSense was nothing but a failure for me. Installing it at multiple clients caused me to have nightmarish drives to fix simple issues (PFSense would refuse to boot if the 2nd WAN port was down, decide it had an Intel NIC and take hours to boot, get wedged, etc).

The real kick in the pants was when they killed UDP multicast forwarding with no warning when upgrading, just boom, IPTV no longer works! We moved to Lede, which has been extremely fault tolerant. I no longer worry that I won't be able to remotely fix a router just cause it had a minor hardware issue.


You also gain extra hardware and protocol support by going to a Linux based solution e.g. BSD does not support 6LoWPAN or any IEEE 802.15.5 hardware to my knowledge.


There are lots of packages that just don't get maintained on BSD, when it comes to gigabit or less routers the whole market for consumer and small/medium business is based on customized versions of OpenWRT from the large router vendors. The depth of maintained packages and documentation is significantly larger.


I love FreeBSD, but as a routing platform, VyOS/EdgeOS are pretty nice.


There's also Router7 recently shown on HN: "Show HN: Router7 – A pure-Go implementation of a small home internet router" [1]

PC Engines APU2 [2] is a good hardware platform, and then there's the more expensive but complete Turris Omnia / Turris Mox.

[1] https://news.ycombinator.com/item?id=17530086

[2] http://www.pcengines.ch/apu2.htm


router7 is really cool.


What's your hardware story?


We had a few netgate boxes, but ran into RAM dying on two of 'em, which would cause them to get wedged iirc (Never ran into wedging outside Netgate boxes). On that particular model the ram was soldered down, making it a nightmare to repair.

On the custom boxes we put in later, it was less hardware issues and much more LTE modem failure (where the LTE modem needed a reboot and wasn't offering a USB ethernet interface, causing PFSense to halt and catch fire).

Today we're split between Ubiquiti's UniFi for our earlier rollouts after dumping PFSense, and Lede for the latter ones as the remote, centralized manageability of both is a huge leap forward from what PFSense ever offered us. I can pull live stats, configure VLANs and implement them remotely and much more.


I looked at pfsense before going with Ubitquti/UniFi. Have to say I'm really happy with it, breeze to setup and does 99% if what you want to do out of the box.


The UniFi suite has some rough edges, the Unifi Security Gateways will just sit there in a useless state if both WANs are down and they are rebooted (until one of the WANs comes up and they connect to the controller). That is a less common scenario, but its still annoying & breaks the local network. Better to run in a crippled state than not at all IMO!


If the blocking issue is a startup connection to its controller, could this be mitigated by having a local controller?


Ouch, that is bad. I guess that will happen pretty much every time there is an power outage.

If internet hasn't recovered all local devices will boot up and not even get an IP?


Na, the Cable/DSL modems and LTE Modems we spec out usually come online a bit before the USG. Ubiquiti's USG and USG Pro take a solid minute to boot, much longer than the modems take. That was another nice feature of Lede, it is booted and ready in well under a minute.


That assumes that internet is working and that all the gear on the way also have had time to boot. That is very much not guaranteed. And if you ever have an internet outage this is probably a likely time to have it.

I also realized that we probably don't have this problem. The controller is in a VM that is pretty much only booted when we need to edit something. So most of the time there won't be any controller to talk to during boot.


cmb the core developer of PFSense went over to Ubiqiti and a new company (Netgate) is now in charge of PFSense.


What’s the takeaway? Are they doing a good job?


labeling cmb "the core developer" is inaccurate. There are many others, but Chris did provide leadership for the project until he left.

There are an easy half dozen senior engineers working on pfSense now.

I'm looking forward to any answers on the "are they doing a good job?" portion. Feedback is a gift.


Love that you are responding on here. It’s great to have your outreach.

Chris and electric sheep fencing sure felt like the core developer for a while.

Clearly he left for a reason. He’s stayed silent about it but rarely does a founder leave without their company being acquired. Feels like he didn’t like the working relationship as it matured.

Personally, I don’t like the change in pricing for support plans that was implemented and I haven’t recommended the products netgate sells since that change. I was helping sell around 10 of your firewalls a year before that.

Nor do I like how OPNSense was treated by netgate. That was dirty stuff.

In any case, you don’t need me to like how you run your business if you have bigger customers that you pivoted to chase. I was small potato’s.


> I was small potato’s

Still important though. Thanks for writing.




Consider applying for YC's Summer 2026 batch! Applications are open till May 4

Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: