FIC forensic challenge validation server

This is a CTF server for distributing and validating exercices. It is design to be robust, so it uses some uncommon technologies like client certificate for authentication, cryptographic functions and DMZ network architecture.

Development And Testing

The easiest way to have a working server is to build a Docker container.

Docker

First, build the container with the following command:

docker build -t fic .

Then, run it with:

docker run -t -i -P fic

It will ask you for a passphrase, you must provide one with at least 4 characters. This key is used to generate the server certificate.

When you see:

root@xxxxxxxxxxxx:/var/www/fic-server#

congratulations, the container is running!

Use docker ps to view to which local ports was assigned the contained webserver.

Database

Demo data are available in /var/www/fic-server/db/feed.sql. In test environment, you can run the following command:

mysql -u root fic < /var/www/fic-server/db/feed.sql

Frontend container

To run the frontend on the same machine as the backend (but in another container), run the following command:

docker run -P -ti --volumes-from BACKEND_CNTNR_NAME FRONTEND_IMG

Production Environnement

Setup

You should compile/install hardened kernel (with latest stable GrSec patch) on each machine.

Prefer GNU/Linux distributions where most packages are compiled with -fPIC and -fstack-protector, like Ubuntu or Gentoo Hardened.

As machines aren't always in safe place (transportation, night before CTF, ...), disks should be encrypted.

Always set strong password when it is possible eg. SSL certificats, ...

Frontend

Keep in mind that this is the machine exposed to participant.

Requirements

• nginx with those modules: aio (for fast delivery of huge content), fastcgi, rewrite, ssl;
• php-fpm with mcrypt module (for submission encryption);

Firewall rules

Expose to participants only 80 and 443 ports.

Expose on synchronization interface the 22 port, used for synchronization and administration purpose from backend.

DROP has to be the default rule for INPUT, FORWARD and OUTPUT chains; use CONNTRACK states.

Backend

Docker containers

Main Docker backend container relies on several other container:

• MySQL database;
• Database storage (as data only container);
• PKI storage;
• PKI shared storage;
• challenge files containers;
• the backend.

To have a fully working backend:

1. Create a data-only container:

docker run --name mysql_data -v /var/lib/mysql busybox

2. Setup the MySQL server:

docker run -d --name db_setup --volumes-from mysql_data -e MYSQL_ROOT_PASSWORD=mysecretpassword -e MYSQL_USER=fic -e MYSQL_PASSWORD=anotherpassword -e MYSQL_DATABASE=fic mysql

Fill the database:

docker build -t db_filler db/
docker run --rm -it --link db_setup:db db_filler

Stop it:

docker stop db_setup
docker rm db_setup

3. Run the database container:

docker run -d --name db --volumes-from mysql_data -e MYSQL_USER=fic -e MYSQL_PASSWORD=anotherpassword -e MYSQL_DATABASE=fic mysql

4. Setup the PKI storages:

docker run --name pki_storage -v /var/www/fic-server/PKI busybox
docker run --name shared_storage -v /var/www/fic-server/shared busybox

5. Build the PKI configuration container:

docker build -t pki_setup pki/

6. Configure the PKI

For development purpose, you can run the default setup:

docker run --rm -it --volumes-from pki_storage --volumes-from shared_storage pki_setup

For production environment:

docker run --rm -it --volumes-from pki_storage --volumes-from shared_storage pki_setup /bin/bash
TODO next steps

7. Build the perl Mcrypt debian package

docker build -t perl-mcrypt perl-mcrypt/
docker run --name mcrypt_builder perl-mcrypt
docker cp mcrypt_builder:$(docker diff mcrypt_builder | grep -oE '[^ ]+deb$') ./
docker rm mcrypt_builder

8. Build and run the backend:

docker build -t backend .
docker run --rm -it --link db:db --volumes-from pki_storage --volumes-from shared_storage -v /home/files:/var/www/fic-server/files-in backend

Where /home/files is your local directory containing all challenge files.

Requirements

• realpath;
• mysql;
• nginx with fastcgi module;
• php-fpm with mysql module;
• openssl and pwgen for client certificat generation;
• mcrypt;
• HTTP::Request::Common perl module (provided by libwww-perl);
• Digest::Whirlpool perl module (provided by lib-digest-whirlpool-perl);
• Mcrypt from CPAN (cpan -i Mcrypt, on Debian, it requires libltdl-dev and build-essential) to decrypt submissions (see https://metacpan.org/pod/Mcrypt);

Files distribution

You need to manually place challenge given files in the tree. To avoid path guessing, files path are hashed. To generate hashed paths, use the script gen_hash_link_files.sh:

mkdir $TO
./gen_hash_link_files.sh FROM TO

Where FROM is the directory with the orignal tree and TO the directory where placed symlink.

Firewall rules

This machine shouldn't have any network connection, except outgoing one to the frontend for synchronization.

Others setups

Indicate in /etc/hosts.conf IP(s) of the frontend.

Run

Two scripts are available, depending if directories synchronization has to be made or not.

You don't need to handle synchronization if it's done by a separate container or if frontend is linked to backend.

The launch.sh and launch_local.sh scripts do all backend stuff for you: synchronization with frontend (only launch.sh), submission checking and smart static pages regeneration.

History

FIC2014

Two machines (DC7900: Core 2 Quad) were used : one for backend (Deimos) and one for frontend (Phobos). They ran a GNU/Linux Gentoo Hardened with custom 3.2 kernel without module loading, unused and unecessary components and with all GrSecurity features activated.

Each machine was two network interfaces: one was used to permit to the backend machine to connect to the frontend (over IPv6). The second interface on the backend was used for administration purpose (with a laptop not connected to Internet). The second interface on the frontend was used to provide network connectivity to participants.

Both frontend and backend were 2 500GB hard-drives with software RAID1. The whole logical RAID disk was LUKS encrypted using Serpent algorithm.

The D Day

Interact with the scheduler

When you launch launch.sh or launch_local.sh script, a socket is open at /tmp/scheduler.sock. Use perl comm-socket.pl /tmp/scheduler.sock to connect to the scheduler. Consult gen_site.pl manual (perldoc gen_site.pl) for list of available instructions.

More

TODO