FIC forensic challenge validation server

This is a CTF server for distributing and validating exercices. It is design to be robust, so it uses some uncommon technologies like client certificate for authentication, cryptographic functions and DMZ network architecture.

Development And Testing

The easiest way to have a working server is to build a Docker container.

Docker

First, build the container with the following command:

docker build -t fic .

Then, run it with:

docker run -t -i -P fic

It will ask you for a passphrase, you must provide one with at least 4 characters. This key is used to generate the server certificate.

When you see:

root@xxxxxxxxxxxx:/var/www/fic-server#

congratulations, the container is running!

Use docker ps to view to which local ports was assigned the contained webserver.

Database

Demo data are available in /var/www/fic-server/db/feed.sql. In test environment, you can run the following command:

mysql -u root fic < /var/www/fic-server/db/feed.sql

Frontend container

To run the frontend on the same machine as the backend (but in another container), run the following command:

docker run -P -ti --volumes-from BACKEND_CNTNR_NAME FRONTEND_IMG

Production Environnement

Setup

You should compile/install hardened kernel (with latest stable GrSec patch) on each machine.

Prefer GNU/Linux distributions where most packages are compiled with -fPIC and -fstack-protector, like Ubuntu or Gentoo Hardened.

As machines aren't always in safe place (transportation, night before CTF, ...), disks should be encrypted.

Always set strong password when it is possible eg. SSL certificats, ...

Backend

Docker containers

Main Docker backend container relies on several other container:

• MySQL database;
• Database storage (as data only container);
• PKI storage;
• PKI shared storage;
• challenge files containers;
• the backend.

To have a fully working backend:

1. Create a data-only container:

docker run --name mysql_data -v /var/lib/mysql busybox

2. Setup the MySQL server:

docker run -d --name db_setup --volumes-from mysql_data -e MYSQL_ROOT_PASSWORD=mysecretpassword -e MYSQL_USER=fic -e MYSQL_PASSWORD=anotherpassword -e MYSQL_DATABASE=fic mysql mysqld --skip-name-resolve

Fill the database:

docker build -t db_filler db/
docker run --rm -it --link db_setup:db db_filler

Stop it:

docker stop db_setup
docker rm db_setup

3. Run the database container:

docker run -d --name db --volumes-from mysql_data -e MYSQL_USER=fic -e MYSQL_PASSWORD=anotherpassword -e MYSQL_DATABASE=fic mysql mysqld --skip-name-resolve

Here, we use the --skip-name-resolve option, because without internet connection, MySQL try to find hostname from IP and hang a lot.

4. Setup the PKI storages:

docker run --name pki_storage -v /var/www/fic-server/PKI busybox
docker run --name shared_storage -v /var/www/fic-server/shared busybox

5. Build the PKI configuration container:

docker build -t pki_setup pki/

6. Configure the PKI

For development purpose, you can run the default setup:

docker run --rm -it --volumes-from pki_storage --volumes-from shared_storage pki_setup

For production environment:

docker run --rm -it --volumes-from pki_storage --volumes-from shared_storage pki_setup IPorURL

Where IPorURL is the way the certificat will authenticate: if challengers will access the frontend with a custom domain, indicate this domain (eg. epita_challenge.fic.local); else indicate the IP of the front host on the network (eg. 192.168.0.5).

For example, 2015 PKI generation looks like:

docker run --rm -it --volumes-from pki_storage --volumes-from shared_storage pki_setup 192.168.0.5

7. Build the perl Mcrypt debian package

docker build -t perl-mcrypt perl-mcrypt/
docker run --name mcrypt_builder perl-mcrypt
docker cp mcrypt_builder:$(docker diff mcrypt_builder | grep -oE '[^ ]+deb$') ./
docker rm mcrypt_builder

8. Build and run the backend:

docker build -t backend .
docker run --rm -it --link db:db --volumes-from pki_storage --volumes-from shared_storage -v /home/files:/var/www/fic-server/files-in backend

Where /home/files is your local directory containing all challenge files.

Requirements

• realpath;
• mysql;
• nginx with fastcgi module;
• php-fpm with mysql module;
• openssl and pwgen for client certificat generation;
• mcrypt;
• HTTP::Request::Common perl module (provided by libwww-perl);
• Digest::Whirlpool perl module (provided by lib-digest-whirlpool-perl);
• Mcrypt from CPAN (cpan -i Mcrypt, on Debian, it requires libltdl-dev and build-essential) to decrypt submissions (see https://metacpan.org/pod/Mcrypt);

Files distribution

You need to manually place challenge given files in the tree. To avoid path guessing, files path are hashed. To generate hashed paths, use the script gen_hash_link_files.sh:

mkdir $TO
./gen_hash_link_files.sh FROM TO

Where FROM is the directory with the orignal tree and TO the directory where placed symlink.

Firewall rules

This machine shouldn't have any network connection, except outgoing one to the frontend for synchronization.

Others setups

Indicate in /etc/hosts.conf IP(s) of the frontend.

Run

Two scripts are available, depending if directories synchronization has to be made or not.

You don't need to handle synchronization if it's done by a separate container or if frontend is linked to backend.

The launch.sh and launch_local.sh scripts do all backend stuff for you: synchronization with frontend (only launch.sh), submission checking and smart static pages regeneration.

Frontend

Keep in mind that this is the machine exposed to participant.

Docker containers

1. Generate the synchronization SSH key on the backend:

sudo su -c "ssh-keygen -t rsa -b 8192 -N '' -f /var/www/fic_server/.ssh/id_rsa" synchro

2. Copy ~synchro/.ssh/id_rsa.pub into front_synchro/authorized_keys file.

3. Run the front_synchro container:

docker build -t synchro front_synchro/
docker run -d --name fsync -p 2242:22 synchro

4. Run the backend.

A first synchronization have to be made before the next step. This synchronization will copy the frontend private key and various required certificates.

5. Run the frontend container:

docker build -t frontend front/
docker run --rm -it -p 80:80 -p 443:443 --volumes-from fsync frontend

Requirements

• nginx with those modules: aio (for fast delivery of huge content), fastcgi, rewrite, ssl;
• php-fpm with mcrypt module (for submission encryption);

Firewall rules

Expose to participants only 80 and 443 ports.

Expose on synchronization interface the 22 port, used for synchronization and administration purpose from backend.

DROP has to be the default rule for INPUT, FORWARD and OUTPUT chains; use CONNTRACK states.

History

FIC2014

Two machines (DC7900: Core 2 Quad) were used : one for backend (Deimos) and one for frontend (Phobos). They ran a GNU/Linux Gentoo Hardened with custom 3.2 kernel without module loading, unused and unecessary components and with all GrSecurity features activated.

Each machine was two network interfaces: one was used to permit to the backend machine to connect to the frontend (over IPv6). The second interface on the backend was used for administration purpose (with a laptop not connected to Internet). The second interface on the frontend was used to provide network connectivity to participants.

Both frontend and backend were 2 500GB hard-drives with software RAID1. The whole logical RAID disk was LUKS encrypted using Serpent algorithm.

The D Day

Interact with the scheduler

When you launch launch.sh or launch_local.sh script, a socket is open at /tmp/scheduler.sock. Use perl comm-socket.pl /tmp/scheduler.sock to connect to the scheduler. Consult gen_site.pl manual (perldoc gen_site.pl) for list of available instructions.

More

TODO