Sunday 14 December 2014

SECURITY : OPENBSD VS FREEBSD




INTRODUCTION :
___________________________________________________
OpenBSD and FreeBSD are both great OS that I admire and use. OpenBSD is considered more secure since it is its main goal, but FreeBSD can be tweaked to be pretty well hardened as well. Depending on the forums or to who we ask, we will have different opinions. But what are the facts? Which OS is more secure and why?

I am not asking the question about which one is globally better, as "better" has a different meaning depending on the context and the needs (ISP routers, database servers, home gateway, desktop system, storage server or appliance, etc...). On some enterprises doing a major OS upgrade every 6 months or every year is doable, on others, it's not possible at all. Also, it depends if one needs performance for streaming (Netflix), or if security is a top priority for a redundant firewall. Everyone needs is different, and both OS are highly useful.

If we strictly focus on security, how does FreeBSD compare to OpenBSD security-wise? In what follows, we will dig into memory protection, system and network security features, and default "out of the box" security. The purpose is to give unbiased facts, to compare point by point both OS. I am not trying to find the "best" OS and discredit the other, I love and use both :-) Let us try to find out the integrated security features of both OS, the visit continues below!



SUMMARY :
___________________________________________________


1. MEMORY PROTECTION
___________________________________________________
The security issue most OS are facing are overflows (buffer, heap). When such a vulnerability exists in an OS service or an application, it can be possible for an attacker to exploit it to make a Denial of Service, or remotely executes its code, a payload. We see an endless discovering of such vulnerabilities witch often ends up in remote code execution. It is a very critical issue, and both OpenBSD and FreeBSD have some defenses against it. Some of the following information is extracted from a Theo de Raadt presentation document as well as other sources.


1.1 - RANDOM STACK GAP

The first problem is that by default, a given program always has its stack at the same address. Therefore, you can find the buffer address and overflow it, with an appropriate exploit. That leads to a very classic buffer overflow, with potential remote code execution (on a network related service for example). The buffer overflow is due to a common programming error, where a bounds check is lacking. For instance filling a 16 bytes buffer with 100 bytes data supplied by an external source (without checking and discarding it). If the supplied data is maliciously constructed, it can inject executable code into the process and take control of it. If the vulnerable process has root/admin rights that's even worse!

The solution implemented into OpenBSD, is to place a variable memory gap before the stack. Consequently, each time a given program runs, its stack will never start at the same address, and thus the buffer has a non fixed address too. This was easily implemented into the kernel, and it renders stack overflows more difficult. It is enabled by default.

I am not aware of a similar random stack gap memory protection on FreeBSD.

1.2 - W^X (WRITE XOR EXECUTE)

Another problem with memory exploits such as buffer overflows, is that the address space has memory that is both writable and executable. Therefore it is possible to write/inject executable code into the vulnerable process stack, and then use the vulnerability to execute that code. If it wasn't possible to execute a writable area, or if it wasn't possible to write to an executable area, exploitation would be much harder.

The solution was implemented into OpenBSD with the W^X policy. The first step to render the stack non executable, is to move away from the stack top page the "signal trempoline" (sigtramp). This move is done at a per-process random address. Now the stack can be rendered non executable. That would seem great, however stopping here would not be enough. Indeed, shared libraries used by the process can still have writable/executable pages. Also, additional objects are executable when they do not require it: Global Offset Table (GOT), Procedure Linkage Table (PLT), c++ constructors (ctors), and c++ destructors (dtors). To fix that, OpenBSD gives GOT and PLT their own non writable page, ctors and dtors move with GOT thus becoming non writable too. At the end, no page is both executable and writable. Another trick is about readonly strings and pointers stored in the .text segment, being executable. They are now moved in the ELF .rodata segment being readonly. Finally, the atexit() function has been modified to not contain writable function pointers anymore. At the end, the overall W^X policy is a beautiful piece of work, and it is enabled by default.

On the FreeBSD side, the non-executable stack is enabled by default for 32bits and 64bits system for ELF, with the sysctl parameters kern.elf32.nxstack=1   and kern.elf64.nxstack=1 . I did not find much information about its implementation, so I guess it just marks the stack as non-executable with the NX bit.

1.3 - ASLR

Exploiting a flaw requires at some point to locate and use a memory address, where a particular piece of code of interest is. ASLR's job is to randomize the position in memory of such area : stack, heap, and libraries. This way, every time a process starts, its libraries won't be loaded at the same address, making the job more difficult for the attacker to predict target adresses. For instance, an exploit such as return-to-libc, which can bypass non-executable stack protection, would be stopped by ASLR as libraries addresses are random.

OpenBSD introduced ASLR and enabled it by default in its system in 2003, and modified it in 2008. It is implemented to randomize shared librairy mapping, base address of mapping, and order of mapping. Each time a process runs, libraries and mapping addresses, and mapping orders, are differents. In 2008, Position Independant Executable (PIE) was added to ASLR. PIE makes the main base program to be randomly loaded as well, as if it was itself a shared librairy. On each run, the main program has a new address.

Another feature used in OpenBSD ASLR is randomized mmap(). This function is a call that maps files or devices into memory. Each time a process makes that call, unless explicitely requiring a fixed address, the result will be a random address, adding yet more memory randomization. A similar feature is random malloc(). This function is used to manually allocate dynamic memory. Tiny malloc() calls are randomly allocated within a special "bucket" page, while larger ones rely on random mmap(). malloc() always randomize, but the addition of guard pages has to be enabled in malloc.conf with the G option. Few malloc() features cannot be enabled by default yet, as some program not well coded are bugging with it. In case it would still not be enough, the free() call delay is also random.

As of December 2014, FreeBSD 10.1-RELEASE has no ASLR available yet. Oliver Pinter started working on ASLR on its side, and Shawn Webb joined him later on to help work on the matter. You can follow their work on GitHub. On March 2014 Shawn Webb has written being hard at work on FreeBSD ASLR implementation. Also, they together have created in 2014 the project HardenedBSD (based off FreeBSD) meant to bring security enhancements to FreeBSD. Shawn Webb highlights that their current ASLR implementation for FreeBSD uses stackgap randomization, which is weak. They are working on true stack randomization to have a stronger ASLR. Also, and it is really worth mentioning, they are actively working on W^X, true stack randomization, and vDSO (virtual Dynamic Shared Object) randomization. All of that work is planned to be upstreamed in FreeBSD.



1.4 - STACK PROTECTOR / STACK SMASHING PROTECTION

Another way to defend against buffer overflows is to introduce what is commonly called a "canary". A canary is a random number/flag inserted on the stack at runtime. Afterwards, this canary is checked, and if the value has changed, it means the stack was overwritten and the program is terminated. OpenBSD Stack Protector implementation also does a reordering on the stack of strings, integers, and pointers. It moves strings closer to the canary, and integers and pointers farther.

In FreeBSD, such memory protection mechanism exists and is called Stack Smashing Protection (SSP) or "ProPolice", where a canary (called "guard") is inserted on the stack. SSP exists since FreeBSD 8.0 but only applied to the base system at first, until very recently (November 2014), which means that third party packages finally also benefit from it (i.e Apache, NGINX, Postfix, MySQL, etc...). SSP also reorders local variables and pointers. Apparently ProPolice have to be enabled with the security.bsd.stack_guard_page=1   sysctl (disabled by default). security.bsd.stack_guard_page sysctl description is "Insert stack guard page ahead of the growable segment", which would match ProPolice behavior.

OpenBSD has an additional mechanism called StackGhost only available on Sparc/Sparc64. It is a weaker stack protector but has no overhead. It uses a unique hardware feature of Sparc architecture.

1.5 - OTHER PROTECTIONS

FreeBSD has a "NULL page mapping" protection which is enabled by default, with the sysctl security.bsd.map_at_zero=0 . This prevents a user to call mmap() with a NULL page. I found no direct mention of mapping at 0/NULL on OpenBSD, but Ted Unangst was kind enough to inform me that OpenBSD generally prohibits mmap at NULL by setting VM_MIN_ADDRESS to a value greater than 0 (which is not configurable).

Then FreeBSD has two other memory corruption detections (rather than protection) aimed at debugging, and disabled by default. The first one is called RedZone and is a Heap Smashing detection aimed at debugging the kernel. RedZone is using a static canary value, and is thus less useful than ProPolice. The other one, MemGuard, is a memory allocator used to detect use-after-free exploits. However it is not compatible with the Universal Memory Allocator used in FreeBSD. OpenBSD memory freeing implementation relies on munmap(), which unmaps the entire allocation. Further access to this freed memory leads to a crash.

OpenBSD also provides since OpenBSD 2.4 strlcpy() and strlcat() : "The strlcpy() and strlcat() functions provide a consistent, unambiguous API to help the programmer write more bullet-proof code". These functions can help developpers avoid certain coding errors leading to buffer overflows. FreeBSD introduced these functions in FreeBSD 3.3.

To sum up the memory protections available on both OpenBSD and FreeBSD:
OpenBSD :
-> Random Stack Gap, W^X (GOT, PLT, ctors, dtors, .rodata, atexit), ASLR (PIE, mmap, malloc), Stack Protector, StackGhost, munmap(), strlcpy()/strlcat()

FreeBSD :
-> NX, ProPolice, NULL page mapping, strlcpy()/strlcat()

FreeBSD's memory protections are all covered by OpenBSD system wide with W^X, Stack Protector, and ASLR. While I use and love both OS, I must admit that the comparison hurts.





2. SYSTEM SECURITY
___________________________________________________
Besides memory protection, system security relies on other areas such as randomness, encryption, privilege separation, and hardening with jail or securelevel for instance. No single element can provide strong security alone, but stacked together, the security gain can be huge. Below we will explore these essential components, and see how our two OS perform in these various areas. The following is based on this OpenBSD slide, another article, and this Sophos NakedSecurity article.

2.1 - RANDOMNESS

Randomness is absolutly vital for the overall security of a system. Indeed, many functions and features rely upon it, such as key generation for encryption algorithms. If the random generator is weak, it can generate "random" numbers which are in fact very predictable. That would mean being able to break into a system by guessing the cryptographic keys needed to enter. Weak randomness dangerously lowers security. The problem is not easy to solve as computers are deterministics, and generating truly random numbers requires to use external resources (ethernet packets, heat, keyboard inputs, HDD writes, sounds, etc...).

One source of randomness is the well known /dev/random device. However it cannot be called from everywhere, like from a chroot, and it is sometimes not wise to use it, like to use a random ID for a network packet. Accessing /dev/random from a locked area, certain libraries, or for every packet is not possible. OpenBSD created the arc4random() function to answer these needs and create strong randomness (pseudo-random generator). Basically, a seed exists as a file on the filesystem, at boot time it is used and mixed with clock sources and available interrupts, then passed to the kernel as an ELF segment, and goes trought the ChaCha20 cipher (instead of RC4, since October 2013). Both rc.script and shutdown script create a new seed, and periodic reseeds are done in between (for instance after a certain amount of time has passed). This randomness is then used accross the whole system for ASLR, mmap(), malloc(), SSP, process ID, thread, packet ID, packet sequence number, /dev/random, NAT, port allocation, etc...

arc4random() is used as well by many other OS, including Android, Blackberry, MacOS, NetBSD, and FreeBSD. The OpenBSD slide linked above however blames FreeBSD RPC xid number generation to be based on an old algorithm where the number is not really random at first, and subsequently incremented. There is not much information about that point, even in the Hackfest 2014: "arc4random - randomization for all occasions" video. It would mean FreeBSD does not use arc4random() for that particular xid generation. Also, while OpenBSD's process ID are random by default, it has to be enabled on FreeBSD with the sysctl kern.randompid   and takes as a value a modulus number, the greater the more random (i.e kern.randompid=1000 ). Finally, while OpenBSD and FreeBSD both use a strong random generator that can be relied on, Ted Unangst from OpenBSD, said in September 2014 "arc4random implementations in FreeBSD and NetBSD aren't quite state of the art anymore". I have no details about that point.

2.2 - ENCRYPTION

Once strong randomness is achieved, we can use it for encryption. On every OS, encryption can be used to encrypt files or communications (SSH, SSL, IPSEC, etc...). Both OpenBSD and FreeBSD are able to do swap encryption, however it is enabled by default only in OpenBSD. To do that on FreeBSD you can follow the security chapter of one of my previous article.

One of the most exciting news from the last months regarding crypto is the birth of LibreSSL, created by OpenBSD (and included in OpenBSD 5.6).



LibreSSL was created to replace the (in)famous OpenSSL crypto and TLS library. OpenSSL has suffered in 2014 a major security bug, the catastrophic Heartbleed. Heartbleed is a vulnerability allowing anyone to remotely read a vulnerable system memory in a non intrusive way, to retrieve secret keys, usernames and passwords. I was able at that time to test a vulnerable system with Metasploit and see with horror the username and password in plaintext at first try... That event motivated OpenBSD team to look into OpenSSL library, and they saw so much non sense (like heartbeat feature only useful for DTLS protocol over UDP, implemented for TLS over TCP) and coding mess, that they decided to fork OpenSSL. More detailed information about why OpenSSL is badly coded on this page.

LibreSSL, while having a cleaner code base and fixing some coding horrors, supports also more ciphers such as Brainpool, Chacha, Poly1305, and ANSSI FRP256v1. There is ports for other OS, but I hope to see it integrated and used by default in FreeBSD in the future, like OpenSSH is.

2.3 - PRIVILEGE SEPARATION

Often a program needs to do a few privileged operations, while all of its other operations can be made with unprivileged user rights. Even if such program needs admin rights just when starting up, for opening a socket for example, it will then keep it until it ends. That means that all of the operations done while it is running will be with higher privileges than needed, which can be very dangerous. Any attacker successfully exploiting such service will gain admin rights on the system.

Then comes the notion of privilege separation. which is best explained with OpenSSH. One way to separate privileges is to create two processes. The parent one with privileged rights, also called the monitor, and the unprivileged child, also called the slave. The child runs as an unused user, and is chrooted in /var/empty. While the parent receives the first connection request, the key exchange and authentification is delegated to the forked child, which communicates with the parent trough a secure interface. The parent decides if the authentication is sucessful or not and informs the child. Once authenticated, another child is forked wih the authenticating user rights, and handles the new SSH session. This way at the most exposed time, when an attacker may try to authenticate, brute force, or exploit SSH, the child has no rights and is chrooted. Above is the simplified version, technically the parent accepting the connection forks a privileged child handling that particular connection, which itself forks an unprivileged one (the rest stay the same as above). More detail in depth in this paper.

OpenBSD by default uses privileges separation for many programs. I found a list on this 2010 document: sshd, bgpd, ntpd, ospfd, dvrmpd, ospf6d, spamd, relayd, ripd, pflogd, snmpd, hostapd, smtpd, tmux, ypldap, ldpd, syslogd, mopd, bind. Other known programs: Postfix and qmail. I can also see on my router that the following programs have both a root process and an unprivileged child : ntpd, sshd, syslogd, pflogd, named, openvpn, and even tcpdump (which is in addition chrooted). It may have many more programs concerned, that is just my router. OpenVPN client has to be configured to run chrooted in its .ovpn file with 3 lines : user _openvpn, group _openvpn, chroot /var/empty (don't forget to mkdir /var/empty/tmp). Dnscrypt-proxy is an example of a program which chroots itself to its user's (_dnscrypt) home directory and drops root privileges (that's called privilege revocation).

On FreeBSD side, the privilege separation is handled by the recent Capsicum feature. It was experimental and optional in FreeBSD 9, and is now enabled by default since FreeBSD 10.0. Capsicum is a sandbox framework from the University of Cambridge Computer Laboratory, supported/funded by Google and the FreeBSD Foundation. Capsicum allows programs to have capability flags, granting them only needed system calls. A capsicum enabled program will only be able to make system calls its capabilities allow it to. For instance if a developper makes an application to read files, he can build capsicum calls in its application to request only file reading/opening capability. Consequently, if the applications misbehaves, no matter because of a bug or an exploit, it won't be able to open a socket or make outbound network connections, terminate other processes, load kernel modules, etc... The application being sandboxed, its file reading requests will be delegated to a trusted component operating outside the sandbox, ensuring privilege separation. Capsicum is used by FreeBSD to harden OpenSSH built-in privilege separation explained above. Capsicum is used in FreeBSD 10.0 by default on the following programs: tcpdump, auditdistd, hastd, dhclient, kdump, rwhod, ctld, iscsid, and uniq. There is a difference however about tcpdump on FreeBSD, which although using Capsicum, is still running as root. An example on how Capsicum is used for dhclient : "It is no longer possible for the unprivileged process to send UDP packets to arbitrary destinations".

Both OpenBSD and FreeBSD take privilege separation seriously.

2.4 - SYSTEM HARDENING

When we have strong memory protection in place, true randomness, cryptography, and privilege separation as a core rule, what can we do to harden the system further? Both OS uses a securelevel in which they run, and depending on the level, from -1 to 2 in OpenBSD, and from -1 to 3 in FreeBSD, various security measures are enforced. OpenBSD runs by default at securelevel 1, whereas FreeBSD runs at the unsecure securelevel -1 (disabled) by default. Extract of interesting security measures enforced by securelevel:

Securelevel 1: "Secure Mode"
- /dev/mem and /dev/kmem may not be written to
- raw disk devices of mounted file systems are read-only
- system immutable and append-only file flags may not be removed
- kernel modules may not be loaded or unloaded
- a panic or trap cannot be forced

Securelevel 2: "Highly secure mode"
- all effects of securelevel 1
- raw disk devices are always read-only whether mounted or not
- settimeofday(2) and clock_settime(2) may not set the time backwards or close to overflow
- firewall and NAT rules may not be altered (available on FreeBSD securelevel 3: "Network secure mode")

Securelevels allow the use of chflags, enabling us to put flags on file to make them immutable (cannot be changed, moved, or deleted), append-only, or nodump (won't be backed up by dump utility unless it's a full backup). The immutable chflags is very powerful as even root cannot modify such protected file (system securelevel has to be lowered first by rebooting in single user mode, to modify /etc/sysctl.conf). In Highly secure mode (or Network secure mode in FreeBSD), the system is so locked down than even firewall rules cannot be altered. It can be a powerful tool to harden a system, as well as a pain to administer such restricted server. It should be used wisely. All in all, both OpenBSD and FreeBSD are able to enforce securelevels, although not enabled by default on FreeBSD.

Then, both OS have different security features: systrace for OpenBSD, and MAC and Jails for FreeBSD. Systrace is a tool similar to AppArmor. You can with it define a policy, stay in learning/interactive mode as long as you wish to adapt your systrace policy, and when you are ready, enforce it. A systrace profile will restrict what system calls a program can make, and therefore which actions a program is allowed to do. You can for example allow a process to make DNS requests to localhost but not outside, access /etc/app.conf but not /etc/other.conf, read a file but not killing a process, etc... Generally, making and maintaining program system call enforcement policies is time consuming and tedious. It is moreover more complicated for big/complex programs, that may sudenly exhibit a new behavior not allowed in the policy, or access a new file or path after an update. On the positive side, if well implemented, a systrace enforced program is a lot harder to exploit and abuse.

While on OpenBSD it is possible to combine chroot and systrace to sandbox a program, FreeBSD provides on its side a feature called Jails. A jail is a separate environnement with its own users and processes, its own IP address and hostname. Only the kernel from the host is shared to the jail. For a complete article about FreeBSD and jails you can check my previous article FreeBSD Gateway Hardening : Jails & Intrusion Detection With Snort. Basically, everything installed and running in the jail is separated from the host and cannot access it. By default a jail is very restricted, and programs inside cannot do a simple ping or sniff the network interface traffic. Jails are a convenient way to sandbox a network related program (i.e web server), in a way more secure than a simple chroot, and without to play with AppAmor or Systrace policies (host is unreachable by default). Of course, no matter the program is in jail or not, on FreeBSD or OpenBSD, it still has to be configured and secured correctly (i.e do not load Apache modules you do not need, set appropriate file permissions, apply updates, etc...). Jails are a very convenient feature.

Then there is the FreeBSD Mandatory Access Control (MAC) framework. MAC framework has not much documentation except the FreeBSD handbook, which got me lost at first read. Basically the MAC framework provides security modules, which can be seen as FreeBSD plugins, that extend the security features of the OS or add new ones.

Let's see a concrete example right now. On the base system, without loading any MAC module, you can use the sysctl security.bsd.see_other_uids=0   to prevent users to see processes and network sockets owned by others. It is useful to isolate users globally, and to prevent a webserver for example to be able to return the entire system processes list to an attacker if exploited. However, if for any reason you want to prevent root to see processes and sockets of other users, it's not directly possible. The solution is to use the mac_seeotheruids.ko MAC module, that can be loaded in /boot/loader.conf   with the line mac_seeotheruids_load="YES". Then, new sysctls appear, security.mac.seeotheruids.enabled=1   (by default) which enables the isolation of users except root (same effect as above without MAC). Also, the sysctl security.mac.seeotheruids.suser_privileged=1   (by default) which can be set to 0 to prevent root seeing other processes and sockets. Other sysctls appear that allow you to fine tune exactly what you want, such as exempting certains groups from this policy (which by the way can be the reason to load this MAC module, not necessarily to restrict root).

Other MAC modules exist. The MAC BSD Extended Policy (mac_bsdextended.ko) allows you to create a filesystem firewall, preventing for instance the account uid X to access the object /your/path. The MAC Port Access Control List Policy (ac_portacl.ko) can be used to allow an unprivileged application to bind to a specific privileged port below 1024 (i.e HTTP). Then finally, the MAC framework can be used to put labels on objects, and give clearances on processes/users. This could create a complex policy where top secret information cannot be accessed by objects not having the required clearances. It can ensure, depending on the MAC module used, that a higher clearance object cannot access, or be accessed by, a lower clearance one.

OpenBSD and FreeBSD both have great hardening features which are highly valuable: securelevel and chflags (both), systrace (OpenBSD), Jails and MAC (FreeBSD). Once again, not a single security feature is bullet proof, but used together with what we have talked about above, it can greatly raise the bar for the attacker.





3. NETWORK SECURITY
___________________________________________________

3.1 - RANDOMNESS

As we have seen in the system security chapter, true randoness is critical to many security areas. Networking is another area which greatly benefits from this randomness pool! Lack of randomness in the network stack has led to many attacks in the past. Even recently (CVE-2014-7284) on 2014 the linux kernel was found on some system to never initialize random seed secret value, used to generate TCP sequence number, IP IDs, and ephemereal ports. Being able to predict some of these numbers, an attacker can hijack a connection, close it, or spoof TCP packets sent to a victim.

On OpenBSD, network randomness is taken seriously. As its randomness is embedded in its pf firewall, and since pf is included on FreeBSD, a FreeBSD system using pf should benefits from the same randomness. FreeBSD's pf version is older than current OpenBSD 5.6 pf version though, more on that later. Network randomness used in pf is pulled from arc4random() we talked before, a reliable randomness source to use in the network stack! To avoid being a predictable network system, randomness is used for: ephemeral source ports, NAT source port, IP IDs, TCP ISN, TCP timestamp, DNS query ID, NTP. NTP protocol by default sends the host localtime to check RTT (Round Trip Time), sending a random value is done instead to avoid information leakage.

Using this randomization and with additional ICMP sequence number verifications, it is possible to block known attacks such as blind connection-reset, blind throughput-reduction (ICMP Source Quench), and blind performance-degrading (ICMP message with "fragmentation needed" + DF bit). This 2005 paper lists OpenBSD as having implemented countermeasures to the 3, while FreeBSD had a 2/3 score, and had an "upcomming" fix for the third attack in 2005 (I hope it's fixed by now, nearly 10 years later). DNS query ID attack is also avoided by the use of pseudo-random DNS query ID and random source port. IPID attack / idle scanning (nmap scan) and TCP fragment injection, are avoided with random IP IDs. NAT deadlock resolution protection requires random source port per destination (ip + port), and TCP ISN modulation. A last issue with extremely busy servers, because of too much TIME_WAIT states, the same {srcip srcport dstip dstport} tuple can be reused, making certain OS to stall affected connections, like XP and FreeBSD (6.x at the time it was said). OpenBSD has applied a patch in 4.4 version. I did not find a patch for FreeBSD by searching "time_wait" in the changelogs of all versions from 6 to 10.1 however, thus it may still be affected.

OpenBSD is clearly proactive in protecting its network stack and improving pf firewall security and reliability. FreeBSD benefits from nearly all of this parameters when using pf, although the security tweaks that has to be done not in pf but in the OS network stack take more time to be implemented into FreeBSD.

3.2 - PF MAGIC

The traffic normalization feature provided by pf (scrub) is a very interesting feature which has broad effects. It is used to handle packet verification, fragments, spoofed traffic, and other malformed packets. It can be used to sanitize a packet before letting it continue its way. It can also be used to enforce a network policy to packets flowing through it. Scrubbing, can:
- enforce a maximum segment size for TCP packets
- enforce a minimum TTL for IP packets
- clear the DF bit of IP packets
- randomize IP ID of packets from hosts having a weak network stack
- statefully normalize TCP [TTL]: no side of a connection is allowed to reduce its TTL
- statefully normalize TCP [Timestamp modulation]: randomize timestamp number
- statefully normalize TCP [Extended PAWS checks]: extend security of TCP sequence number (10->18 bits).

Fragment reassembly is on by default as its advantage (security increase) outweights its downside (memory use). Indeed, when there is no reassembly, only the first fragment contains the necessary headers to be checked against filtering rules, other fragments won't match the rules. No state entries can be created for fragments too. Also, I have seen attacks in the past where the first fragment was forged with the IP headers to "look good", as to fool a firewall, while the fully reconstructed packet could in fact connect to a different port (Overlapping Fragment Attack). It was also possible to crash an OS when it was reassembling the malformed packet (Teardrop Attack). Various fragment attacks explained on this page.

pf can also detect and block spoofed packets with the built-in keyword "antispoof". Instead of creating yourself rules to drop network addresses that cannot be seen on X interface, and create other rules to drop spoofed traffic on Y interface, you can use this simple feature. In one line, pf can block any spoofed packet coming to a given interface. If your LAN network is changed in the future, the ruleset won't need to be updated. It really simplifies the ruleset and spares you doing the job. Likewize, the "urpf-failed" feature can check for you that a packet coming to an interface has an address (i.e 10.0.0.145) the interface passed on (i.e 10.0.0.1) is the correct path to this address (that's a Unicast Reverse Path Forwading).

pf has some other features I won't necessarily detail. Quickly, "synproxy" can protect a server behind the firewall from a SYN flood attack. OS fingerprinting can be added to rules to make more custom rules to allow only certains OS (ex : OpenBSD to the SSH port of the router). You can use "route-to" to route a packet to an interface/path different than the default gateway for instance. You can tag packets to create policy based filtering and create trusted flows, etc...

On both OpenBSD and FreeBSD we can use pf, and therefore have access to all of these features. However, as I mentioned earlier, FreeBSD 10.1 has an old pf version, from OpenBSD 4.5 (2009). Currently, OpenBSD is at version 5.6, which means 11 versions later (5 years). This is very unfortunate as in pf versions above 4.5, between 4.6 and 4.9, performances greatly improve whereas before 4.6 under high load performances collapse. Also, new keywords, features, and syntax was introduced after 4.5, such as the "match" and "divert-to" keywords. A lot of IPv6 fixes and improvements were done, scrub modification, NAT rewrite, and various fixes. Below is a quick sample of few modifications I have choosen to show.

From 4.6 to 5.6, modifications (22 selected modifications, there is a lot more):
4.6 - icmp tracking code rewritten (shotcomings found there)
4.6 - scrub modification
4.6 - "match" keywork
4.7 - NAT rewrite
4.7 - "divert-to" keywork
4.9 - log subsystem rewritten for performance and features
5.0 - Make sure IPv6 packets with routing headers do not create state while dropping them in pf(4).
5.0 - Fixed crash in pf(4) ioctl(2).
5.0 - Fixed potential null dereference in pf(4) ioctl, ahd(4).
5.0 - Added IPv6 ACK prioritization in pf(4).
5.0 - Cleaned up protocol checksums in pf(4), IPv4 and MPLS.
5.0 - Make pf(4) reassemble IPv6 fragments.
5.1 - Improve pf(4) ICMPv6 direction check.
5.1 - pf(4)s IPv6 code evolves further. [...] to make the code more robust.
5.1 - Fix a pf(4) bug where pf_walk_option6() used the outer header in the pd2 case.
5.3 - Lower pf.conf(5) frags limit. Avoids running out of mbuf clusters when dealing with lots of IP fragments.
5.3 - Fixed pf(4) sloppy state tracking missing half the connection in asymmetric setups and ignoring state match in icmp(4) direction checks.
5.4 - Do not reset the pf(4) fragment timeout each time a fragment arrives; drop all fragments if the packet cannot be reassembled within 60 seconds.
5.4 - Before pulling TCP options from the mbuf onto the stack, do an additional length check in pf(4) so overflow cannot happen.
5.5 - Resolved an issue where icmp(4) traffic with pf(4) nat-to failed due to incorrect checksums.
5.5 - Fixed pf(4) icmpid bug (only affected icmp(4) echos via nat, when the nat doesn't change the address).
5.6 - Fixed path MTU discovery with ping6(8) through pf(4) using nat or rdr.

We can see that given all the IPv6 work and fixes, that the current FreeBSD pf version seems broken regarding IPv6. Current OpenBSD pf is bleeding edge, whereas FreeBSD version is a very old version. I know this fact has started debates on the Internet, as Henning Brauer said that current pf was 4 times faster than older versions. FreeBSD has patched its version to be multithreaded, and from what I have read it would be too much work to rework a completely different pf version such as 5.6. You can read this thread Future of pf in FreeBSD ? - does it have one ?. I originally heard about this legitimate interrogation at BSDnow.





4. DEFAULT SECURITY
___________________________________________________
There is two opposite stances toward default security. The first considers that it doesn't matter a security feature is enabled by default or not, as long it can be enabled (usually done once when the server is installed, and never touched again). The second one considers that having to enable everything will lead to mistakes, options can be forgotten or wrongly configured. Also, the more security features and options you want to enable, the more an expert you need to be. When it's on by default, you gain a lot of time, you make no mistakes, and you do not need to be an expert. I usually adhere more to the first logic, although I recognize that the second one is completely relevant. Using both OpenBSD and FreeBSD I must admit that secure by default, when it touches so many complex areas we have seen above, should be the gold standard.

4.1 - OUT OF THE BOX

Below is a graphical overwiew of what we have seen so far, with MANY CAVEATS thought:
- graphics can be interpreted to make them said opposite things
- I made choices while doing it, but if I did it otherwise I could have made looked better either OS
- I could have added metrics or score, which do not accurately represent a whole OS security
- I added colors to make the reading easier, not to make OS looks good or bad
- having security tools like systrace or MAC is great, but if you don't know how to use them they are useless
- I have chosen to privilege secured by default, but I could have colored in green available security even if disabled by default
- it is only about security, not the whole OS (i.e administration, performance, package install, update/upgrade, hardware support, community, etc...)
- may be you don't care for Sparc64 (and hence for memory protection for this OS)
- graphics are cool and fun, however security being a whole process it cannot be sume up with "green" or "red"
- graphics, by being an overview, are lacking of details (i.e OpenBSD's use-after-free protection with munmap() not present in FreeBSD)

That being said, here it is:



Someone at the back of the room seems to protest, what? What are you saying? .../me dodging flying eggs... Ah ok I understand, no matter it is disabled by default as long as we can enable it. .../me dodging other eggs!...) Ok you don't care for Sparc64, got that. You know building LibreSSL on your system and deal with all linked packages? pf 4.5 is good enough for you? Ok ok, here you go, don't hit me :-)



If we enable all disabled by default protections, either with sysctls (random process ID, SSP), or with manual steps (swap encryption, securelevel, LibreSSL port), if we don't use Sparc nor IPv6, and we disregard bits here and here that are not perfect (RPC xid, arc4random() implementation), the final security could perfectly fits your needs. I am providing both point of view, it's up to you.

4.2 - SOURCE CODE AUDIT

OpenBSD developpers constantly have between 6 to 12 of them doing code auditing, instead of writing new code or features. Code auditing allows for finding bugs, or sometimes new class of bugs. This way of working puts a great priority on code quality, that ultimately ends as better security. Indeed, only adding new features and programs and never looking back until a security advisory knocks at your door, is not the best way to make a secure OS.

Most security problems are in fact in the first place simple bugs. When facing a bug it is not always obvious if it can be exploitable and create a vulnerability. In that last case, it can even be harder to gauge the exploitation feasability/probability. What is simple however, is to fix any bug found. This way, if it is later discovered to lead to a security vulnerability, your OS will already be fixed before the problem even exists! Also, when a kind of bug or even a new class of bug is discovered, the code previously audited is audited again looking for this new coding mistake. When not proactively looked for and fixed, bugs that create critical vulnerabilities can exist for years before being discovered:
- WordPress had an XSS flaw for 4 years, which could allow an attacker to execute administrative actions if the website administrator loaded a page containing the rogue comment/code.
- Heartbleed bug has existed for 2 years, and certain people suspect that intelligence agencies were well aware and using it
- POODLE SSLv3 vulnerability has existed for 18 years (since 1996) and can be used to read encrypted communications
- A Windows vulnerability remotely exploitable existed for 19 years and could be used to take over any Windows computer
- The LZO compression algorithm had an integer overflow for 20 years. It is used on many software such as OpenVPN for instance. This flaw may lead to remote code excution under certain conditions.
- Another OpenSSL flaw (CVE-2014-0224) discovered in 2014 existed for 16 years, and could be used to read TLS encrypted communications.
- Shellshock bug existed for 25 years and can be used in many different systems and programs, such as webservers. Even OpenVPN was affected by it.

The above examples show that currently, in any OS, bugs which are exploitable exist. If they are left untouched, they won't magically fix themselves, and will eventually become critical vulnerabilities allowing remote exploitation for 20 years or more! As you may have already noticed, that's not because we do not publicily know a vulnerability, that it has not been known and secretely used by others.

Proactively and aggressively auditing its source code for bugs is a very smart way to dramatically increase security, and is one of OpenBSD's strongpoint being part of the "Secure by default" philosophy.





CONCLUSION :
___________________________________________________
If we consider memory protection only, OpenBSD is the master in its area (Random Stack Gap, W^X (GOT, PLT, ctors, dtors, .rodata, atexit()), ASLR (PIE, mmap(), malloc()), Stack Protector, StackGhost, strlcpy()/strlcat()), and FreeBSD is severely lagging behind. I hope projects such as HardenedBSD will bring ASLR and more to FreeBSD as soon as possible. Memory exploitation is probably the most common remote exploitation technique, it is not a subject to be trifled with.

Then regarding system hardening, both OpenBSD and FreeBSD are interesting in this regard, in the way they provide different tools to do it. Both can make use of securelevel and chflags, make use of strong randomness, and cryptography. OpenBSD is more privilege separation by default + systrace, while FreeBSD provides also privilege separation with Capsicum, and adds the MAC framework and the famous and very useful jails. The pf firewall is amazing, and I hope FreeBSD will follow more regularily OpenBSD pf versions instead of staying 5 years behind.

With all of that said, which OS is the most secure? Which one to choose? These are not easy questions. Indeed, if we focus only on memory protection, OpenBSD seems to be the way to go. However, if you like or need jails, you won't have any on OpenBSD (although you can use systrace instead, which is not the same thing). At the opposite, you could say that you are aware of the weaker memory protection on FreeBSD, but that you will use jails to isolate your network service. Then we can wonder what would be the best between 1) a system where memory exploitation is very hard and has less chances of success (very low probability of host compromise), and 2) a system where memory exploitation is more probable but the host out of reach because the exploitable service is jailed. That alone will highly vary depending on the software installed and associated risks (a single Postfix VS Apache + MySQL + PHPMyAdmin + Drupal, or a webserver VS a router). Another point of view, is if your webserver is compromised, with all client data, it does not matter if the host will be compromised or not, it's already lost (in that case you may prioritize memory protection with OpenBSD). If you add up other parameters such as ease of administration, LTS versions and support, team work (everyone should know/learn the new system in your team), the "best" OS will be highly individual!



Security is one important parameter, especially nowadays where the Internet is more like an unmerciful war zone than anything else. Vulnerabilities with far reaching consequences hit us many times per year, and everyone is hacking everyone (countries and states included). However, you should understand the OS you choose, and keep it simple. Simplicity is one key to security, and is one area where OpenBSD shines with its secure by default design. However it could be argued that making a jail is easier and stronger than creating and maintaining a systrace policy, and if you perfectly understand jails but not sure about systrace, choose FreeBSD. There is many arguments to prove that both OS are the best! (in no particular order and not necessarily about security: ZFS, SSD, memory protection, jails, pf, journaled-softupdates filesystem, systrace, capsicum, MAC, etc...).

In the end both OpenBSD and FreeBSD have their strongpoints and disadvantages. I really wish both will continue to improve and eventually borrow from each other the best ideas. Projects like LibreSSL or HardenedBSD need our support to improve, don't hesitate to help if you have the possibility.

I wish you a more secure and peaceful 2015 (soon) year! Thank you for reading :-)

updated December 16 2014: FreeBSD SSP, OpenBSD malloc/NULL, random free(), HardenedBSD additional information, default/tweaked graphics


LINKS
___________________________________________________


Saturday 29 November 2014

FreeBSD Gateway Hardening : Jails & Intrusion Detection With Snort




INTRODUCTION :
___________________________________________________________
In my previous article Defend Your Network And Privacy : VPN Gateway With FreeBSD, we made a FreeBSD router acting as a VPN gateway, and providing DNS cache and encryption, and DHCP. It was further hardened by restricting users privileges, increasing memory protection, adding randomness in the IP stack and system processes, enabling swap encryption, creating an SSH Eliptic Curve key, and increasing securelevel. However, there was still some room for improvement.

One of the greatest FreeBSD's features is Jails. Jails enable us to create a separate environnement, with its own users and processes, its own IP address and hostname, that cannot access the running host system. A jail is especially useful for a service exposed to the internet such as a webserver, but at some extent for any network service. Indeed jailing a DHCP server for instance may not prevent an exploit from a malicious client on the LAN side, but it will at least not give automatically full takeover over the host. A jail is not meant to prevent a criminal from craking the service running inside it, indeed a jail cannot do anything about the last MySQL or PHP vulnerability, but rather will contain the intruder once he gets in. It is an additional strong security layer an intruder will have to try breaking to access the host, and it will give you time to spot the intrusion. It can make all the difference between the hack of a network service running on the host with a complete takeover of the running host and the remaining network, and between an intrusion confined into a jail, unable to access the host or to do lateral moves on the network. Jails are not a bullet proof invincible security feature however, but they still are very strong if properly used.

In the following we will harden our FreeBSD gateway a little more. We will move into jails the DNS cache Unbound, as well as DNSCrypt-proxy. While we are at it, we will move into another jail the DHCP server, and finally in a third jail, we will install and configure the Network Intrusion Detections System (NIDS) Snort.



DESCRIPTION :
___________________________________________________________
The FreeBSD gateway we will harden is structured as below :
- (jailed) DHCP server to distribute network parameters to your LAN
- (jailed) DNS cache/server to speed up DNS requests
- (jailed) DNS encryption so that every outgoing request is encrypted on port 443
- (jailed) Network Intrusion Detection System (Snort) to warn you about suspicious traffic
- (host) VPN gateway to your VPN provider


It has been validated and working for the FreeBSD VPN gateway configured in my previous article, but it should be applicable to any FreeBSD router.


PREREQUISITES :
___________________________________________________________
- A working FreeBSD router, preferably FreeBSD 10.1-RELEASE or better.
- A warm cup of coffee (I always function better with it!)

The network map is a typical home network with one or two computers, and one DSL router from the ISP :


I am using the usual router, a Shuttle DS437, which is a fanless low energy router (10W idle). FreeBSD 10.1-RELEASE finally properly recognise the network card chipset RTL8111G! That's a good news, it is no longer required to use a STABLE or CURRENT version.




SUMMARY :
___________________________________________________________


1. CREATING JAILS
___________________________________________________________
There is different ways to build a jail. You can use a package dedicated to jail building and management such as ezjail, or build jails from sources manually. If the latter, you can unpack sources from the CD installation media, or download them using subversion package. We will use this last method. If you want an example with ezjail + DHCP + Bind, check this blog.

Also, for jails to work properly, no process on the host should listen on all interfaces (usually *:port). On the default FreeBSD install, NTP and Syslog are concerned. Please check this procedure if not already done, before going forward.

We have to choose a path for building our jail, it can be for instance /usr/jail/jail-dns, as it will prepare the next jail we will setup. Then we install subversion, and download the sources:
$ DIR=/usr/jail/jail-dns
$ sudo mkdir -p $DIR

$ sudo pkg install subversion
$ svn checkout http://svn0.eu.freebsd.org/base/release/10.1.0 /usr/src

Once it's done, we can build our jail. For the first jail it takes a long time to do the buildworld step, but this step has to be done only once:
$ cd /usr/src
$ sudo make buildworld

$ sudo make installworld DESTDIR=$DIR
$ sudo make distribution DESTDIR=$DIR
$ sudo mount -t devfs devfs $DIR/dev

Once the jail building process is done, there is a minimal set of files to create, or copy from the host, for the jail to run:
$ sudo cp /etc/localtime $DIR/etc/
$ sudo cp /etc/resolv.conf $DIR/etc/
$ touch $DIR/etc/fstab

Create a minimal /etc/rc.conf:
$ sudo vi $DIR/etc/rc.conf
# IP not defined here, but on the host
network_interfaces=""

# Sendmail : disabled
sendmail_enable="NO"
sendmail_submit_enable="NO"
sendmail_outbound_enable="NO"
sendmail_msp_queue_enable="NO"

# Syslog : do not open network socket
syslogd_flags="-ss"

At this point, to build further jails, some steps can be skipped (like buildworld). It is basically as follow:
$ DIR=/usr/jail/jail-xxx
$ sudo mkdir -p $DIR
$ cd /usr/src
$ sudo make installworld DESTDIR=$DIR
$ sudo make distribution DESTDIR=$DIR
$ sudo mount -t devfs devfs $DIR/dev

Do not forget to add a minimal /etc/rc.conf as well. The whole process is easily scriptable, and more friendly to use when scripted. Now that at least a jail is created, let us continue to the next step to configure our DNS jail.




2. JAIL-DNS
___________________________________________________________
As our first step, we need to define the jail's IP address (10.0.0.2) and hostname (jail-dns), and enable jails use in /etc/rc.conf. We also need to add the jail's IP address as an alias for our LAN network interface re0:
$ sudo vi /etc/rc.conf
# Jails
jail_enable="YES"
ifconfig_re0_alias0="inet 10.0.0.2 netmask 255.255.255.0"

As the re0 alias will be created at next boot, if we want to use it now without rebooting, we have to create it:
$ sudo ifconfig re0 10.0.0.2 alias

The last main configuration file for jail, /etc/jail.conf:
$ sudo vi /etc/jail.conf
exec.start = "/bin/sh /etc/rc";
exec.stop = "/bin/sh /etc/rc.shutdown";
exec.clean;
mount.devfs;

path = /usr/jail/$name;
host.hostname = $name;

jail-dns {
ip4.addr = 10.0.0.2;
securelevel=3;
}

Everything above the line "jail-dns {" is global and effective for every jail. The variable "$name" is automatically populated with the jail name to which it applies, which means that if we have many jails, it will be applied to each jail using the jail's name. Of course if you want a jail hostname different than the jail name, you can move the "host.hostname = your-name" inside the individual jail config between the brackets. About the securelevel, if you do not specify it, the jail default securelevel will be the same as the host. If the host has a securelevel equals to 1, the jail will have the same. You can comment it out at first, and once your jail is fully working, set it to 3 at the end.

Now let's start our jail and check it is running:
$ sudo service jail start
$ jls
JID IP Address Hostname Path
1 10.0.0.2 jail-dns /usr/jail/jail-dns


First things to do once your jail is live, rebuild the database mail aliases, change root password, and eventually create an additional unprivileged user. Accounts inside the jail are only jail's account. For instance Jail's root account is not the host's root account and has no power on the host. It is advised to set a different password in the jail. We can also update it before going further:
$ sudo jexec jail-dns newaliases
$ sudo jexec jail-dns passwd
$ sudo jexec jail-dns adduser

$ sudo freebsd-update -b /usr/jail/jail-dns fetch install

We need to enable the DNS cache Unbound, which is part of the base system. We also need to install the package dnscrypt-proxy. To install a package, you can either drop a shell into the jail, or install it from the host as below:
$ sudo pkg -j jail-dns install dnscrypt-proxy

Either way, we will continue from within the jail (notice the prompt going from $ to #):
$ sudo jexec jail-dns sh
# hostname
jail-dns

Setting up Unbound and Dnscrypt startup parameters (you can choose the DNS server of your choice):
# vi /etc/rc.conf
# DNS
dnscrypt_proxy_enable="YES"
dnscrypt_proxy_resolver="dnscrypt.eu-dk"
dnscrypt_proxy_flags='-a 10.0.0.2:40'
local_unbound_enable="YES"

Create Unbound file unbound.conf:
# vi /etc/unbound/unbound.conf
server:
username: unbound
directory: /var/unbound
chroot: /var/unbound
pidfile: /var/run/local_unbound.pid
auto-trust-anchor-file: /var/unbound/root.key
do-not-query-localhost: no
interface: 10.0.0.2
access-control: 10.0.0.1/24 allow

include: /etc/unbound/forward.conf

Create Unbound file forward.conf:
# vi /etc/unbound/forward.conf
forward-zone:
name: .
forward-addr: 10.0.0.2@40

Modify /etc/resolv.conf to not point to the host anymore, but on our own IP address instead:
# vi /etc/resolv.conf
options edns0
nameserver 10.0.0.2

Prevent any automatic file configuration modification by the system:
# chflags uchg /etc/unbound/forward.conf
# chflags uchg /etc/unbound/unbound.conf
# chflags uchg /etc/resolv.conf

Start Unbound and DNSCrypt, and check it is working with a randomly chosen DNS name:
# service dnscrypt-proxy start
# service local_unbound start

# host www.bsdnow.tv
www.bsdnow.tv has address 65.39.148.220

Log off from the jail, modify the /etc/resolv.conf of the host to point to your jail-dns, and if working disable your host Unbound and DNSCrypt if already installed:
# CTRL + D
$ sudo vi /etc/resolv.conf
options edns0
nameserver 10.0.0.2

$ host www.bsdnow.tv
www.bsdnow.tv has address 65.39.148.220

$ sudo service local_unbound stop
$ sudo service dnscrypt-proxy stop

$ host www.bsdnow.tv
www.bsdnow.tv has address 65.39.148.220

$ sudo vi /etc/rc.conf
# Comment all existing DNS entries related to Unbound and DNSCrypt
#dnscrypt_proxy_enable="YES"
#dnscrypt_proxy_resolver="dnscrypt.eu-dk"
#dnscrypt_proxy_flags='-a 127.0.0.1:40'
#local_unbound_enable="YES"

Eventually delete dnscrypt-proxy from the host if installed:
$ sudo pkg delete dnscrypt-proxy
$ sudo pkg autoremove

Our first jail is operational.




3. JAIL-DHCP
___________________________________________________________
Follow again the steps in CREATING JAILS to create jail-dhcp at /usr/jail/jail-dhcp

We now want to create a DHCP server jail with the IP address 10.0.0.3 and hostname jail-dhcp. We again need to add the jail's IP address as an alias for our LAN network interface re0:
$ sudo vi /etc/rc.conf
# Jails
jail_enable="YES"
ifconfig_re0_alias0="inet 10.0.0.2 netmask 255.255.255.0"
ifconfig_re0_alias1="inet 10.0.0.3 netmask 255.255.255.0"

Creating the alias:
$ sudo ifconfig re0 10.0.0.3 alias

DHCP needs full access to the network interface to operate, which is restricted by default in jails. To unlock proper access to network interfaces, we have to modify /etc/devfs.conf:
$ sudo vi /etc/devfs.conf
[devfsrules_unhide_bpf=5]
add path 'bpf*' unhide

[devfsrules_jail_dhcp=6]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add include $devfsrules_unhide_bpf

This enables us to create the jail in /etc/jail.conf:
$ sudo vi /etc/jail.conf
jail-dhcp {
ip4.addr = 10.0.0.3;
securelevel=3;
devfs_ruleset = 6;
}

Now let's start our jail and check it is running:
$ sudo service jail restart
$ jls
JID IP Address Hostname Path
3 10.0.0.2 jail-dns /usr/jail/jail-dns
4 10.0.0.3 jail-dhcp /usr/jail/jail-dhcp


First things to do once your jail is live, rebuild the database mail aliases, change root password, eventually create an additional unprivileged user, and update the jail:
$ sudo jexec jail-dhcp newaliases
$ sudo jexec jail-dhcp passwd
$ sudo jexec jail-dhcp adduser

$ sudo freebsd-update -b /usr/jail/jail-dhcp fetch install

We install the package isc-dhcp42-server from the host as below:
$ sudo pkg -j jail-dhcp install isc-dhcp42-server

We continue from within the jail:
$ sudo jexec jail-dhcp sh
# hostname
jail-dhcp

Main DHCP configuration:
# vi /usr/local/etc/dhcpd.conf
option domain-name-servers 10.0.0.2;

subnet 10.0.0.0 netmask 255.255.255.0 {
option routers 10.0.0.1;
range 10.0.0.100 10.0.0.200;
}

Finally we have to modify /etc/rc.conf to enable dhcp server at startup, and specify additional system configuration. We do not enable chroot, as we are already in jail:
# vi /etc/rc.conf
# DHCP
dhcpd_enable="YES"
dhcpd_flags="-q"
dhcpd_conf="/usr/local/etc/dhcpd.conf"
dhcpd_ifaces="re0"
dhcpd_withumask="022"
dhcpd_chuser_enable="YES"
dhcpd_withuser="dhcpd"
dhcpd_withgroup="dhcpd"
dhcpd_chroot_enable="NO"
dhcpd_devfs_enable="YES"

Let's start our server:
# service isc-dhcpd start
# CTRL + D
$ sudo sockstat -4
dhcpd dhcpd 40216 6 udp4 10.0.0.3:67 *:*
dhcpd dhcpd 40216 20 udp4 10.0.0.3:32984 *:*


Check with a client computer on your LAN that you get an IP address, gateway (10.0.0.1), and DNS server (10.0.0.2), from 10.0.0.3.




3. JAIL-SNORT
___________________________________________________________
Follow again the steps in CREATING JAILS to create jail-snort at /usr/jail/jail-snort

Snort is a Network Intrusion Detection System (NIDS) analysing your network flow, looking for suspicious or known bad traffic (malware, intruder, information leak, malformed packets, etc...). Snort is only useful if you check its logs, and needs tweaking at first to eliminate false prositives pertaining to your network. There is less documentation about running Snort in jail on FreeBSD 10.x. Fortunately, once you know how to configure Snort on the host, it is not very hard to make it running inside a jail. Let's see how to proceed.

We want a Snort jail with the IP address 10.0.0.4 and hostname jail-snort:
$ sudo vi /etc/rc.conf
# Jails
jail_enable="YES"
ifconfig_re0_alias0="inet 10.0.0.2 netmask 255.255.255.0"
ifconfig_re0_alias1="inet 10.0.0.3 netmask 255.255.255.0"
ifconfig_re0_alias2="inet 10.0.0.4 netmask 255.255.255.0"

Creating the alias:
$ sudo ifconfig re0 10.0.0.4 alias

Snort, like DHCP, needs full access to the network interface to operate:
$ sudo vi /etc/devfs.conf
[devfsrules_jail_snort=7]
add include $devfsrules_hide_all
add include $devfsrules_unhide_basic
add include $devfsrules_unhide_login
add include $devfsrules_unhide_bpf

Time to create the jail in /etc/jail.conf:
$ sudo vi /etc/jail.conf
jail-snort {
ip4.addr = 10.0.0.4;
securelevel=3;
devfs_ruleset = 7;
}

Now let's start our jail and check it is running:
$ sudo service jail restart
$ jls
JID IP Address Hostname Path
5 10.0.0.2 jail-dns /usr/jail/jail-dns
6 10.0.0.3 jail-dhcp /usr/jail/jail-dhcp
7 10.0.0.4 jail-snort /usr/jail/jail-snort


Same as before:
$ sudo jexec jail-snort newaliases
$ sudo jexec jail-snort passwd
$ sudo jexec jail-snort adduser

$ sudo freebsd-update -b /usr/jail/jail-snort fetch install

We install the package snort:
$ sudo pkg -j jail-snort install snort

We continue from within the jail:
$ sudo jexec jail-snort sh
# hostname
jail-snort

We will create various folders and files needed for Snort to start:
# mkdir /usr/local/etc/snort/rules/iplists
# touch /usr/local/etc/snort/rules/iplists/default.blacklist
# touch /usr/local/etc/snort/rules/white_list.rules
# touch /usr/local/etc/snort/rules/black_list.rules

Now, you have to create a free account at https://www.snort.org/. You will be able in your account profile to get an "oinkcode", needed to retrieve the Snort registered rules, the real Snort core ruleset. The free oinkcode gives you access to rules 30 days old, while a paid subscription (30$/year for personnal use) gives you access to the latest ruleset. Pick the free one for testing, and once operational if you are serious about intrusion detection, the paid subscription sounds reasonable. The Community ruleset (a single file) does not require an oinkcode.

Once you have your oinkcode, we can configure and use Pulledpork by uncommenting the lines below:
# cp /usr/local/etc/pulledpork/pulledpork.conf.sample /usr/local/etc/pulledpork/pulledpork.conf
# vi /usr/local/etc/pulledpork/pulledpork.conf
# Below replace -oinkcode- with your oinkcode (i.e 455d6c4a5s666s4s478f5s66... without -)
# Registered Ruleset
rule_url=https://www.snort.org/reg-rules/|snortrules-snapshot.tar.gz|-oinkcode-

# NEW Community ruleset:
rule_url=https://s3.amazonaws.com/snort-org/www/rules/community/|community-rules.tar.gz|Community

# NEW For IP Blacklisting! Note the format is urltofile|IPBLACKLIST|-oinkcode-
# This format MUST be followed to let pulledpork know that this is a blacklist
rule_url=http://labs.snort.org/feeds/ip-filter.blf|IPBLACKLIST|-oinkcode-

Then run pulledpork:
# /usr/local/bin/pulledpork.pl -k -c /usr/local/etc/pulledpork/pulledpork.conf -K /usr/local/etc/snort/rules/ -o /usr/local/etc/snort/rules/

If successful, you may consider adding it to crontab to update your rules periodically (i.e : every morning at 10AM):
# crontab -e
* 10 * * * /usr/local/bin/pulledpork.pl -k -c /usr/local/etc/pulledpork/pulledpork.conf -K /usr/local/etc/snort/rules/ -o /usr/local/etc/snort/rules/

It's time to configure Snort main configuration file snort.conf. The full file is not included, only parts that need to be modified:
# vi /usr/local/etc/snort/snort.conf
ipvar HOME_NET [10.0.0.0/24]
ipvar DNS_SERVERS [10.0.0.2]

dynamicpreprocessor directory /usr/local/lib/snort_dynamicpreprocessor/
dynamicengine /usr/local/lib/snort_dynamicengine/libsf_engine.so

#include $PREPROC_RULE_PATH/sensitive-data.rules
var WHITE_LIST_PATH ./rules
var BLACK_LIST_PATH ./rules

output alert_unified2: filename snort.alert, limit 128, nostamp
output log_unified2: filename snort.log, limit 128, nostamp

# site specific rules
include $RULE_PATH/local.rules
include $RULE_PATH/community.rules
include $RULE_PATH/VRT-app-detect.rules
include $RULE_PATH/VRT-blacklist.rules
include $RULE_PATH/VRT-browser-chrome.rules
include $RULE_PATH/VRT-browser-firefox.rules
include $RULE_PATH/VRT-browser-ie.rules
include $RULE_PATH/VRT-browser-other.rules
include $RULE_PATH/VRT-browser-plugins.rules
include $RULE_PATH/VRT-browser-webkit.rules
include $RULE_PATH/VRT-content-replace.rules
include $RULE_PATH/VRT-decoder.rules
include $RULE_PATH/VRT-dos.rules
include $RULE_PATH/VRT-exploit-kit.rules
include $RULE_PATH/VRT-file-executable.rules
include $RULE_PATH/VRT-file-flash.rules
include $RULE_PATH/VRT-file-identify.rules
include $RULE_PATH/VRT-file-image.rules
include $RULE_PATH/VRT-file-java.rules
include $RULE_PATH/VRT-file-multimedia.rules
include $RULE_PATH/VRT-file-office.rules
include $RULE_PATH/VRT-file-other.rules
include $RULE_PATH/VRT-file-pdf.rules
include $RULE_PATH/VRT-indicator-compromise.rules
include $RULE_PATH/VRT-indicator-obfuscation.rules
include $RULE_PATH/VRT-indicator-scan.rules
include $RULE_PATH/VRT-indicator-shellcode.rules
include $RULE_PATH/VRT-malware-backdoor.rules
include $RULE_PATH/VRT-malware-cnc.rules
include $RULE_PATH/VRT-malware-other.rules
include $RULE_PATH/VRT-malware-tools.rules
include $RULE_PATH/VRT-netbios.rules
include $RULE_PATH/VRT-os-linux.rules
include $RULE_PATH/VRT-os-mobile.rules
include $RULE_PATH/VRT-os-other.rules
include $RULE_PATH/VRT-os-solaris.rules
include $RULE_PATH/VRT-os-windows.rules
include $RULE_PATH/VRT-policy-multimedia.rules
include $RULE_PATH/VRT-policy-other.rules
include $RULE_PATH/VRT-policy-social.rules
include $RULE_PATH/VRT-policy-spam.rules
include $RULE_PATH/VRT-preprocessor.rules
include $RULE_PATH/VRT-protocol-dns.rules
include $RULE_PATH/VRT-protocol-finger.rules
include $RULE_PATH/VRT-protocol-ftp.rules
include $RULE_PATH/VRT-protocol-icmp.rules
include $RULE_PATH/VRT-protocol-imap.rules
include $RULE_PATH/VRT-protocol-nntp.rules
include $RULE_PATH/VRT-protocol-pop.rules
include $RULE_PATH/VRT-protocol-rpc.rules
include $RULE_PATH/VRT-protocol-scada.rules
include $RULE_PATH/VRT-protocol-services.rules
include $RULE_PATH/VRT-protocol-snmp.rules
include $RULE_PATH/VRT-protocol-telnet.rules
include $RULE_PATH/VRT-protocol-tftp.rules
include $RULE_PATH/VRT-protocol-voip.rules
include $RULE_PATH/VRT-pua-adware.rules
include $RULE_PATH/VRT-pua-other.rules
include $RULE_PATH/VRT-pua-p2p.rules
include $RULE_PATH/VRT-pua-toolbars.rules
include $RULE_PATH/VRT-scada.rules
include $RULE_PATH/VRT-sensitive-data.rules
include $RULE_PATH/VRT-server-apache.rules
include $RULE_PATH/VRT-server-iis.rules
include $RULE_PATH/VRT-server-mail.rules
include $RULE_PATH/VRT-server-mssql.rules
include $RULE_PATH/VRT-server-mysql.rules
include $RULE_PATH/VRT-server-oracle.rules
include $RULE_PATH/VRT-server-other.rules
include $RULE_PATH/VRT-server-samba.rules
include $RULE_PATH/VRT-server-webapp.rules
include $RULE_PATH/VRT-sql.rules
include $RULE_PATH/VRT-x11.rules

Then we will add a custom rule to be able to test snort's alert log:
# vi /usr/local/etc/snort/local.rules
# Test rule : any ping will trigger it
alert icmp any any -> any any (msg:"ICMP test"; sid:10000001;)

Now test Snort configuration with the -T parameter:
# snort -A console -q -c /usr/local/etc/snort/snort.conf -i re0 -T

If there is no error, start Snort in console mode to display on screen what it is doing:
# snort -A console -q -c /usr/local/etc/snort/snort.conf -i re0

On a LAN computer, do a ping to an external website like google for instance, and check your console. Then stop Snort with CTRL+C, and check the file /var/log/snort/alert:
# vi /var/log/snort/alert
11/28-19:30:57.065893 [**] [1:10000001:0] ICMP test [**] [Priority: 0] {ICMP} 10.0.0.53 -> 64.233.182.94
11/28-19:30:58.066761 [**] [1:10000001:0] ICMP test [**] [Priority: 0] {ICMP} 10.0.0.53 -> 64.233.182.94

It's working. Edit back your local.rules file and comment your ICMP rule. Now we can allow snort to start when the jail initializes:
# vi /etc/rc.conf
# Snort
snort_enable="YES"
snort_flags="-A fast -b -D"

We can start Snort normally as any other service:
# service snort start

When Snort starts, the active memory goes from 5-7MB to 128MB on my router, then goes down to an avereage of 20MB later. From now on you will probably have false positives, and there is a file to take care of them. Let's imagine we have the following alert we want to suppress:

... [**] [138:5:1] SENSITIVE-DATA Email Addresses [**] ... {TCP} 10.0.0.101:55471 -> ...

You can see in bold that this rule can be identified by "138:5". Now all you have to do is to edit the file threshold.conf:
# vi /usr/local/etc/snort/threshold.conf
suppress gen_id 138, sig_id 5, track by_src, ip 10.0.0.0/24

Below is a full example. Do not copy/paste them directly as these may not trigger false positives on your network:
### ID 119 ###
# ----------------------------------------------------------
# Suppress Oversized request URI directory
suppress gen_id 119, sig_id 15, track by_src, ip 10.0.0.0/24

# Suppress Long Header
suppress gen_id 119, sig_id 19, track by_src, ip 10.0.0.0/24

# Suppress Unknown Header
suppress gen_id 119, sig_id 31, track by_src, ip 10.0.0.0/24

# Suppress Unescaped Space in HTTP URI
suppress gen_id 119, sig_id 33, track by_src, ip 10.0.0.0/24

### ID 120 ###
# ----------------------------------------------------------
# Suppress No Content Length or Transfer Encoding in HTTP Response
suppress gen_id 120, sig_id 3, track by_dst, ip 10.0.0.0/24

### ID 128 ###
# ----------------------------------------------------------
# Suppress (spp_ssh) Protocol mismatch
suppress gen_id 128, sig_id 4, track by_src, ip 10.0.0.0/24

### ID 129 ###
# ----------------------------------------------------------
# Suppress Bad Segment
suppress gen_id 129, sig_id 5, track by_src, ip 10.0.0.0/24

# Suppress Consecutive TCP small segments exceeding threshold
suppress gen_id 129, sig_id 12, track by_src, ip 10.0.0.0/24

# Suppress Reset outside window
suppress gen_id 129, sig_id 15, track by_src, ip 10.0.0.0/24

# Suppress TCP session without 3-way handshake
suppress gen_id 129, sig_id 20, track by_src, ip 10.0.0.0/24

A restart is needed for the exclusions to take effect. When in doubt about the remote IP address in an alert, do a Whois on it to help you figure out if it's suspicious or expected traffic.




CONCLUSION :
___________________________________________________________
Snort is cool stuff, above all when you can use professionnal rules for free (30 days old though). It is located at a very privileged network point where all traffic flows both ways, and is therefore in a unique position to catch malicious data. It's free, easier to setup with tools like pulledpork, and can be greatly improved with email notification, and with a web based interface.

Jailing most important network related services, provides strong additional security layers! However it also comes with trade-off, with an host and three jails, we have four systems to keep updated (that means running freebsd-update four times). Also, when upgrading for instance from FreeBSD 10.1 to 10.x, once the host is upgraded, every jail will have to be upgraded from sources as well. Then, jails are very rectricted by default, but by granting more accesses to the network interfaces as we did for DHCP and Snort, we increase the risks. Indeed if a jail with such right is hacked, there is no need for the intruder to break out of the jail, he can just sniff out all of your traffic. Lastly, if a jail is well restricted, but is allowed to access everything in the firewall rules, the security is greatly diminished. Security is a whole process, with user restrictions, folder and file rights, OS hardening, updates, firewall rules, NIDS, etc... Jails are only a piece of it, they should not be underestimated, neither should they be overestimated.

See you for my next article! :-)


LINKS
___________________________________________________________