Data

Tools

Indexes

Applications

Experiments

Awesome

An open API service indexing awesome lists of open source software.

https://github.com/linux-application-whitelisting/fapolicyd

File Access Policy Daemon
https://github.com/linux-application-whitelisting/fapolicyd

access-control linux security whitelisting

Last synced: 23 days ago
JSON representation

File Access Policy Daemon

Awesome Lists containing this project

README 

          
File Access Policy Daemon

=========================



[![Build Status](https://travis-ci.com/linux-application-whitelisting/fapolicyd.svg?branch=master)](https://travis-ci.com/linux-application-whitelisting/fapolicyd)



This is a simple application whitelisting daemon for Linux.



RUNTIME DEPENDENCIES

--------------------

* kernel >= 4.20 (Must support FANOTIFY_OPEN_EXEC_PERM. See [1] below.)



BUILDING

--------



See [BUILD.md](./BUILD.md) for build-time dependencies and instructions for building.



POLICIES

--------

The current design for policy is that it is split up into units of rules

that are designed to work together. They are copied into /etc/fapolicyd/rules.d/

When the service starts, the systemd service file runs fagenrules which

assembles the units of rules into a comprehensive policy. The policy is

evaluated from top to bottom with the first match winning. You can see the

assembled policy by running 



```

fapolicyd-cli --list

```

Originally, there were 2 policies shipped, known-libs and restrictive.



The restrictive policy was designed with these goals in mind:



1. No bypass of security by executing programs via ld.so.

2. Anything requesting execution must be trusted.

3. Elf binaries, python, and shell scripts are enabled for trusted

   applications/libraries.

4. Other languages are not allowed or must be enabled.



It can be recreated by copying the following policy units into rules.d.

The optional ones are not included unless they are needed:



20-dracut.rules

21-updaters.rules

30-patterns.rules

40-bad-elf.rules

41-shared-obj.rules

43-known-elf.rules

71-known-python.rules

72-shell.rules

optional: 73-known-perl.rules

optional: 74-known-ocaml.rules

optional: 75-known-php.rules

optional: 76-known-ruby.rules

optional: 77-known-lua.rules

90-deny-execute.rules

95-allow-open.rules



The known-libs policy (default) was designed with these goals in mind:



1. No bypass of security by executing programs via ld.so.

2. Anything requesting execution must be trusted.

3. Any library or interpreted application or module must be trusted.

4. Everything else is not allowed.



It can be created by copying the following policy units into rules.d:



10-languages.rules

20-dracut.rules

21-updaters.rules

30-patterns.rules

40-bad-elf.rules

41-shared-obj.rules

42-trusted-elf.rules

70-trusted-lang.rules

72-shell.rules

90-deny-execute.rules

95-allow-open.rules



EXPERIMENTING

-------------

You can test by starting the daemon from the command line. Before starting

the daemon, cp /usr/bin/ls /usr/bin/my-ls just to setup for testing. When

testing new policy, its highly recommended to use the permissive mode to

make sure nothing bad happens. It really is not too hard to deadlock your

system. Continuing on with the tutorial, as root start the daemon as follows:



```

/usr/sbin/fapolicyd --permissive --debug

```

Then in another window do the following:



1. /usr/lib64/ld-2.29.so /usr/bin/ls

2. my-ls

3. run a python file in your home directory.

4. run a program from /tmp



In permissive + debug mode you will see dec=deny which means

"decision is to deny". But the program will actually be allowed to run.



You can run the daemon from the command line with --debug-deny command

line option. This culls the event notification to only print the denials.

If this is running cleanly, then you can remove the --permissive option

and get true denials. Now retest above steps and see the difference.



DEBUG MODE

----------

In debug mode, you will see events such as this:



```

rule:9 dec=deny_audit perm=execute auid=1001 pid=14137 exe=/usr/bin/bash : file=/home/joe/my-ls ftype=application/x-executable

```



What this is saying is rule 9 made the ultimate Decision that was followed.

The Decision is to deny access and create an audit event. The subject is the

user that logged in as user id 1001. The subject's process id that is trying

to perform an action is 14137. The current executable that the subject is

using is bash. Bash wanted permission to execute /home/joe/my-ls which is the

object. And the object is an ELF executable.



Sometimes you want to list out the rules to see what rule 9 might be. You can

easily do that by running:



```

fapolicyd-cli --list

```



Also, in fapolicyd.conf, there is a configuration option, syslog_format, which

can be modified to output information the way you want to see it. So, if you

think auid in uninteresting you can delete it. If you want to see the device

information for the file being accessed, you can add it. You can also enable

this information to go to syslog by changing the rules to not say audit, but

instead have syslog or log appended to the allow or deny decision.



OVERRIDE MOUNTS WHILE DEBUGGING

-------------------------------

When debugging you can specify an alternative mounts file to the deamon

to watch for event notifications. This allows for finer level of control

than is achievable by filtering by filesystem type.



The alternative mounts file will expect the same format as `/proc/mounts`,

which allows us to select entries from `/proc/mounts` into a new file which

fapolicyd will use as the mount source.



For example, use grep to select a single mount point:

```

mount -t tmpfs tmpfs /tmp/my-test-dir 

grep my-test-dir /proc/mounts > /tmp/my-test-mounts

fapolicyd --debug --mounts=/tmp/my-test-mounts

```



Here we mount a tmpfs for testing in `/tmp`, and grep it from `/proc/mounts`

into the overriding mounts file, then run fapolicyd in debug mode while

specifying the override file. The result is fapolicyd only receives events

that occur in `/tmp/my-test-dir`.



WRITING RULES

-------------

The authoritative source is the fapolicyd.rules man page.



It is suggested that people use the known-libs set of rules. This set of

rules is designed to allow anything that is trusted to execute. It's

design is to stay out of your way as much as possible. All that you need

to do is add unpackaged but trusted files to the trust database. See the

"Managing Trust" section for more.



But if you had to write rules, they follow a simple

"decision permission subject : object" recipe. The idea is to write a

couple things that you want to allow, and then deny everything else. For

example, this is how shared libraries are handled:



```

allow perm=open all : ftype=application/x-sharedlib trust=1

deny_audit perm=open all : ftype=application/x-sharedlib

```



What this says is let any program open shared libraries if the library

being opened is trusted. Otherwise, deny with an audit event any open of

untrusted libraries.



First and foremost, fapolicyd rules are based on trust relationships.

It is not meant to be an access control system of Mandatory Access Control

Policy. But you can do that. It is not recommended to do this except when

necessary. Every rule that is added has to potentially be evaluated - which

delays the decision.



If you needed to allow admins access to ping, but deny it to everyone 

else, you could do that with the following rules:



```

allow perm=any gid=wheel : trust=1 path=/usr/bin/ping

deny_audit perm=execute all : trust=1 path=/usr/bin/ping

```



You can similarly do this for trusted users that have to execute things in

the home dir. You can create a trusted_user group, add them the group,

and then write a rule allowing them to execute from their home dir.



When you want to use user or group name (as a string). You have to guarantee

that these names were correctly resolved. In case of systemd, you need to add

a new after target 'After=nss-user-lookup.target'.

To achieve that you can use `systemctl edit --full fapolicyd`,

uncomment the respective line and save the change.



```

allow perm=any gid=trusted_user : ftype=%languages dir=/home

deny_audit perm=any all : ftype=%languages dir=/home

```



One thing to point out, if you have lists of things that you would like to

allow, use the macro set support as illustrated in this last example. This puts

the list into a sorted AVL tree so that searching is cut to a minimum number

of compares.



One last note, the rule engine is a first match wins system. If you are adding

rules to allow something but it gets denied by a rule higher up, then move

your rule above the thing that denies it. But again, if you are writing rules

to allow execution, you should reconsider if adding the programs to the trust

database is better.



RULE ORDERING

-------------

Starting with 1.1, the rules should be placed in a rules.d directory under

the fapolicyd configuration directory. There is a new utility, fagenrules,

which will compile the rules into a single file, compiled.rules, and place the

resulting file in the main config directory.



If you want to migrate your existing rules, just move them as is to the rules.d

directory. You cannot have both compiled.rules and fapolicyd.rules. The

fagenrules will notice this and put a warning in syslog. If you use fapolicyd-cli --list, it will also notice and warn. If you do have both files, the old rules

file will be used instead of the new one.



This new rules.d directory is intended to make it easier to develop application

specific rules that can be dropped off by automation / orchestration. This

should make managing the configuration easier.



The files in the rules.d directory are processed in a specific order. See the

[rules.d README](rules.d/README-rules) file for more information.



REPORT

------

On shutdown the daemon will write an object access report to

/var/log/fapolicyd-access.log. The report is from oldest access to newest.

Timestamps are not included because that would be a severe performance hit.

The report gives some basic forensic information about what was being accessed.



PERFORMANCE

-----------

When a program opens a file or calls execve, that thread has to wait for 

fapolicyd to make a decision. To make a decision, fapolicyd has to lookup

information about the process and the file being accessed. Each system call

fapolicyd has to make slows down the system.  



To speed things up, fapolicyd caches everything it looks up so that

subsequent access uses the cache rather than looking things up from

scratch. But the cache is only so big. You are in control of it, though.

You can make both subject and object caches bigger. When the program ends,

it will output some performance statistic like this into

/var/log/fapolicyd-access.log or the screen:



```

Permissive: false

q_size: 640

Inter-thread max queue depth 7

Allowed accesses: 70397

Denied accesses: 4

Trust database max pages: 14848

Trust database pages in use: 10792 (72%)



Subject cache size: 1549

Subject slots in use: 369 (23%)

Subject hits: 70032

Subject misses: 455

Subject evictions: 86 (0%)



Object cache size: 8191

Object slots in use: 6936 (84%)

Object hits: 63465

Object misses: 17964

Object evictions: 11028 (17%)



```



In this report, you can see that the internal request queue maxed out at 7.

This means that the daemon had at most 7 threads/processes waiting. This

shows that it got a little backed up but was handling requests pretty quick.

If this number were big, like more than 200, then increasing the q_size may

be necessary.



Another statistic worth looking at is the hits to evictions ratio. When a

request has nowhere to put information, it has to evict something to make

room. This is done by a LRU cache which naturally determines what's not

getting used and makes it's memory available for re-use.



In the above statistics, the subject hit ratio was 95%. The object cache was

not quite as lucky. For it, we get a hit ration of 79%. This is still good,

but could be better. This would suggest that for the workload on that system,

the cache could be a little bigger. If the number used for the cache size is

a prime number, you will get less cache churn due to collisions than if it

had a common denominator. Some primes you might consider for cache size are:

2039, 4099, 6143, 8191, 10243, 12281, 16381, 20483, 24571, 28669, 32687,

40961, 49157, 57347, 65353, etc.



This report can be scheduled to be written periodically by setting the

configuration option `report_interval`. This option is set to `0` by default

which disables the reporting interval. A positive value for this option

specifies the number of seconds to wait between reports.



Also, it should be mentioned that the more rules in the policy, the more

rules it will have to iterate over to make a decision. As for the system

performance impact, this is very workload dependent. For a typical desktop

scenario, you won't notice it's running. A system that opens lots of random

files for short periods of time will have more impact.



Another configuration option that can affect performance is the integrity

setting. If this is set to sha256, then every miss in the object cache will

cause a hash to be calculated on the file being accessed. One trade-off would

be to use size checking rather than sha256. This is not as secure, but it is

an option if performance is problematic.



## Use ignore_mounts with great care



Starting with fapolicyd-1.3.8, there is a new performance option, ignore_mounts. ignore_mounts removes selected mount points from fanotify monitoring to reduce overhead on very busy filesystems (for example, cache or logging partitions). This improves performance **at the cost of visibility**: when a mount is ignored, fapolicyd does not see opens/reads from that tree and cannot apply policy decisions there.



### When to consider it



+ High-churn **data-only** mounts where monitoring provides little value (e.g. cache directories, file/content serving directories, or dedicated logging partitions).



+ Workloads that are **well understood and controlled**, with correct ownership/permissions/SELinux labels and no expectation that content will be treated as code.



### Risks



+ **Interpreter and plugin gaps**: Even with noexec, trusted interpreters (shell, Python, Java, Node.js, etc.) and applications that load plugins/bytecode may read and act on files from the ignored mount. Those accesses bypass fapolicyd because the mount is not watched.



+ **Policy blind spots**: Content copied into the ignored tree won’t be evaluated while it resides there and may only be detected after it moves to a monitored location.



+ **Coverage assumptions**: The root filesystem / is always monitored. Do not use ignore_mounts to work around denials for native ELF binaries; that is not its purpose.



### Required guardrails



+ Each ignored mount **must** be mounted with noexec. This prevents direct ELF execve() from that mount. If noexec is missing, / is still monitored and the first observed event will be the runtime linker which will trigger the ld_so pattern detection which will deny access. Due to this certain denial, fapolicyd refuses to ignore mount points not mounted with noexec. Mounting with noexec does not mitigate interpreter/JIT/plugin scenarios.



+ **Advisory pre-check** before changing configuration:



```

    fapolicyd-cli --check-ignore_mounts[=MOUNT] [--verbose]

```



 This verifies the mount exists, confirms noexec, scans for files matching the %languages macro (interpreter-consumable content), reports findings, and returns non-zero when suspicious files are found so automation can gate configuration changes.



+ Add entries exactly as shown in the second column of /proc/mounts. Whitespace around comma-separated entries is ignored.



### Conflicts and notes



+ / is always monitored.



+ Do not combine this option with allow_filesystem_mark=1; the daemon will refuse the configuration.



MEMORY USAGE

------------

Fapolicyd uses lmdb as its trust database. The database has very fast

performance because it uses the kernel virtual memory system to put the

whole database in memory. If the database is sized wrongly, then fapolicyd

will reserve too much memory. Don't worry too much about this. The kernel is

very smart and doesn't actually allocate the memory unless its used. However,

we'd like to get it right sized.



Starting with the 0.9.3 version of fapolicyd, statistics about the database

is output when the program shuts down. On my system, it looks like this:



```

Database max pages: 9728

Database pages in use: 7314 (75%)

```



This also correlates to the following setting in the fapolicyd.conf file:



```

db_max_size = 38

```



This size is in megabytes. So, if you take that and multiply by 1024 * 1024,

we get 39845888. A page of memory is defined as 4096. So, if we divide

max_size by the page size, we get 9728 which matches the setting. Each entry

in the lmdb database is 512 bytes. So, for each 4k page, we can have data on

8 trusted files.



An ideal size for the database is for the statistics to come up around 75% in

case you decide to install new software some day. The formula is 



```

 (db_max_size x percentage in use) / desired percentage = new db_max_size

```



So, working with example numbers, suppose max_size is 160 and it says it was

68% occupied. That is wasting a little space. Putting the numbers in the

formula, we get  (160 x 68) / 75 = 145.



If you have an embedded system and are not using rpm. But instead use the file

trust source and you have a list of files, then your calculation is very

different. Suppose for the sake of discussion, you have 317 files that are

trusted. We take that number and divide by 8. We'll round that up to 40. Take

that number and multiply by 100 and divide by 75. We come up with 53.33. So,

let's call it 54. This is how many pages is needed. Turning that into real

memory, it's 216K. One megabyte is the smallest allocation, so you would set

```

db_max_size = 1

```



Starting with the 0.9.4 release, the rpm backend filters most files in the

 /usr/share directory. It keeps anything with a with a python extension or

a libexec directory. It also keeps /usr/src/kernel so that Akmod can still

build drivers on a kernel update.



TROUBLESHOOTING

---------------

Whatever you do, DO NOT TRY TO ATTACH WITH PTRACE. Ptrace attachment sends

a SIGSTOP which cannot be blocked. Since your whole system depends on

fapolicyd approving access to glibc and various critical libraries, that

will not happen until SIGCONT is sent. The system can deadlock if the

continue signal is not sent. Using gdb will have the same results. With

that in mind, let's talk about troubleshooting steps...



If you are using deny_audit and you are not getting any audit events, the

fix is to add 1 audit rule. It can be a rule about anything. Watches tend

to be the highest performance, so maybe just add a watch for writes to

etc shadow and restart the audit daemon so the rule gets loaded.



```

-w /etc/shadow -p w

```



When fapolicyd blocks something, it will generate an audit event if the

Decision is deny_audit and it has been compiled with the auditing option.

The audit system must have at least 1 audit rule loaded to create the full

FANOTIFY event. It doesn't matter what rule. To see if you have any denials,

you can run:



```

ausearch --start today -m fanotify --raw | aureport --file --summary



File Summary Report

===========================

total  file

===========================

16  /sbin/ldconfig

1  /home/joe/./my-ls

```



You can also see which executables are involved like this:



```

ausearch --start today -m fanotify -f /sbin/ldconfig --raw | aureport -x --summary



Executable Summary Report

=================================

total  file

=================================

16  /usr/bin/python3.7

```



However, you probably want to know the rule that is blocking it. Unfortunately

the audit system cannot tell you this unless you are using the 6.3 kernel or

later. What you can do is change the decisions to deny_log. This will write

the event to syslog as well as the audit log. In syslog, you will have the

same output as the debug mode.



The shipped rules expect that everything installed is in the trust database.

If you have installed anything by unzipping it or untarring it, then you need

to add the executables, libraries, and modules to the trust database. See the

MANAGING THE FILE TRUST SOURCE section for instructions on how to do this.



You can ask fapolicyd to include the trust information by adding trust to the

end of the syslog_format configuration option. The things that you need to know

to debug the policy is:



* The rule triggering

* The executable accessing the file

* The object file type

* The trust value



Look at the rule that triggered and see if it makes sense that it triggered. If

the rule is a catch all denial, then check if the file is in the trust db. To see the rule that is being triggered, either reproduce the problem with the daemon running in debug-deny mode or change the rules from deny_audit to deny_syslog. If you choose this method, the denials will go into syslog. To see them run:

```

journalctl -b -u fapolicyd.service

```

to list out any events since boot by the fapolicyd service.



Starting with 1.1, fapolicyd-cli includes some diagnostic capabilities.



|         Option         |              What it does                  |

|------------------------|--------------------------------------------|

| --check-config         | Opens fapolicyd.conf and parses it to see if there are any syntax errors in the file.                     |

| --check-path           | Check that every file in $PATH is in the trustdb. (New in 1.1.5)                                          |

| --check-status         | Output internal metrics kept by the daemon. (New in 1.1.4)                                                |

| --check-trustdb        | Check the trustdb against the files on disk to look for mismatches that will cause problems at run time.  |

| --check-watch_fs       | Check the mounted file systems against the watch_fs daemon config entry to determine if any file systems need to be added to the configuration.                                           |

| --check-ignore_mounts  | Check the configured mounts that are ignored to see that they are mounted noexec and there are no suspicious files in the partition. (New in 1.4)                                       |

| --test-filter          | Test a path to a file against the filter rules to determine if a file will be trusted. (New in 1.3.7)     |



MANAGING TRUST

--------------

Fapolicyd use lmdb as a backend database for its trusted software list. You

can find this database in /var/lib/fapolicyd/. This list gets updated

whenever packages are installed by dnf or rpm by a rpm plugin.



The files that go into the trust database from rpm go through a filter to

eliminate as many unimportant files as possible so that the trust database

is concise. To see what kinds of files are in the trust database, you can try this:



```

fapolicyd-cli -D  | awk '{print $2}'  | awk -F/ '{

     base=$NF

     ext="*"

     if (base !~ /^\./ && base ~ /\./) {

       n=split(base,a,"."); ext="*."a[n]

     }

     path=""

     for(i=1;i "/sys/kernel/security/ipe/Ex Policy/active"

```



and so on. Since they can all be disabled, the fact that an admin can issue a

service stop command is not a unique weakness.



6) How do you prevent race conditions on startup? Can something execute before

the daemon takes control?



One of the design goals is to take control before users can login. Users are

the main problem being addressed. They can pip install apps to the home dir

or do other things an admin may wish to prevent. Only root can install things

that run before login. And again, root can change the rules or turn off the

daemon.



Another design goal is to prevent malicious apps from running. Suppose someone

guesses your password and they login to your account. Perhaps they wish to

ransomware your home dir. The app they try to run is not known to the system

and will be stopped. Or suppose there is an exploitable service on your system.

The attacker is lucky enough to pop a shell. Now they want to download

privilege escalation tools or perhaps an LD_PRELOAD key logger. Since neither

of these are in the trust database, they won't be allowed to run.



This is really about stopping escalation or exploitation before the attacker

can gain any advantage to install root kits. If we can do that, UEFI secure

boot can make sure no other problems exist during boot.



Wrt to the second question being asked, fapolicyd starts very early in the

boot process and startup is very fast. It's running well before other login

daemons.



NOTES

-----

* It's highly recommended to run in permissive mode while you are testing the

daemon's policy.



* Stracing the fapolicyd daemon WILL DEADLOCK THE SYSTEM.



* About shell script restrictions...there's not much difference between

running a script or someone typing things in by hand. The aim at this

point is to check that any program it calls meets the policy.



* Some interpreters do not immediately read all lines of input. Rather, they

read content as needed until they get to end of file. This means that if they

do stuff like networking or sleeping or anything that takes time, someone with

the privileges to modify the file can add to it after the file's integrity has

been checked. This is not unique to fapolicyd, it's simply how things work.

Make sure that trusted file permissions are not excessive so that no unexpected

file content modifications can occur.



* If for some reason rpm database errors are detected, you may need to do

the following:



```

1. db_verify /var/lib/rpm/Packages

if OK, then

2. rm -f /var/lib/rpm/__db*

3. rpm --rebuilddb

```



[1] - https://git.kernel.org/pub/scm/linux/kernel/git/jack/linux-fs.git/commit/?id=66917a3130f218dcef9eeab4fd11a71cd00cd7c9