Mastersync privilege separation

Status

This work is done. All code is merged in the master branch.

Goal

Mitigate exploitation of any bugs in the bundled C/C++ code that ships with NodeJS and can be reached over the network (V8, libuv, OpenSSL etc.).

How

Principle of least privilege, split privileges between system processes:

• chroot to /var/empty*
• drop to a system account that is only used for one specific type of process/privilege
• make sure any other environmental leaks are closed like open file descriptors etc.
• limit the remote attack vector to a single unnamed pipe (the IPC channel with the privileged parent)
• further reduce the attack surface by utilizing an extremely simple authentication protocol:
  • use a simple, easy to parse and bounds check "auth" packet (max 1024 bytes)
  • accept exactly one auth packet per connection
  • don't accept noise or multiple packets, disconnect if not everything is 100% correct
  • disconnect on invalid credentials
  • require only two states per connection, either "authenticated" or "to-authenticate"

Design

Mastersync is split into three different type of processes, each with it's own set of privileges:

• privileged (vs)
  • read password database
  • read oplog
  • fork unprivileged and VC privileged children
  • authenticate and authorize clients coming in via the unprivileged child
• unprivileged (preauth)
  • handle incoming connections (over TCP or Unix domain socket)
  • handle auth requests (parse incoming data as a line-separated JSON stream)
  • pass auth requests to the privileged parent for verification
• VC privilege (vc)
  • write to local database
  • send requested data as BSON over authenticated incoming connections

The privileged process is started as root. It opens an admin connection to the database, spawns VC children and an unprivileged child to handle incoming connections. Once all this is done the privileged process chroots itself and drops privileges to the ms user and group. An IPC channel is kept open during the whole lifetime of the process to each spawned process. Each spawned process chroots and drops privileges right after all needed modules are included.

The privileged process creates a new OplogReader for each spawned Versioned Collection. The oplog reader emits raw BSON documents and is piped to the VC instance via IPC (nodejs uses socketpair(2)).

The privileged process uses it's IPC channel with the unprivileged connection handler to get new auth requests and connections. The privileged process does the verification (bcrypt) and if successful it passes a push request and the connection to the accompanying VC privileged process using it's IPC channel. The VC privileged process can then start sending it's data over the received connection.

This mitigates:

• Remote exploits of bugs in V8, libuv, OpenSSL and other bundled C/C++ code that comes with NodeJS. The code that is reachable over the network is decoupled and chrooted and can only communicate with a privileged parent via a socketpair. Only a strictly typed single JSON object is accepted as input on this channel which makes it easy to parse, put bounds checks in place, and disconnect early if invalid.
• Escalation of the unprivileged process reading or writing to any database, since it has no database connection.
• VC privileged process reading anything from other databases via the oplog, since it gets it's oplog objects over stdin and does not have database credentials to access the oplog by itself.
• Both the privileged process and the unprivileged process can be chrooted to /var/empty. Only VC privileged processes might need file system access, depending on application specific hooks or can be chrooted to /var/empty as well.

TODO

• determine attack surface when using the net module
• determine attack surface when using socketpair for IPC (child_process)
• find a way to prevent race conditions when starting a world writable UNIX domain socket, see discussion
• better handle DoS of preauth process (maybe spawn one preauth per connection)
• determine if different node processes running under the same user can read each others memory

Discussion

https://groups.google.com/forum/#!topic/nodejs/6PDd8xU89xA

---

Inspired by OpenSSH privilege separation: http://www.citi.umich.edu/u/provos/papers/privsep.pdf