server/README.md

FIC forensic challenge validation server
========================================

This is a CTF server for distributing and validating exercices. It is design to
be robust, so it uses some uncommon technologies like client certificate for
authentication, cryptographic functions and DMZ network architecture.

Development And Testing
-----------------------

The easiest way to have a working server is to build a Docker container.

### Docker

First, build the container with the following command:

```
docker build -t fic .
```

Then, run it with:

```
docker run -t -i -P fic
```

It will ask you for a passphrase, you must provide one with at least 4
characters. This key is used to generate the server certificate.

When you see:

```
root@xxxxxxxxxxxx:/var/www/fic-server#
```

congratulations, the container is running!

Use `docker ps` to view to which local ports was assigned the contained
webserver.


### Database

Demo data are available in `/var/www/fic-server/db/feed.sql`. In test
environment, you can run the following command:

    mysql -u root fic < /var/www/fic-server/db/feed.sql


### Frontend container

To run the frontend on the same machine as the backend (but in another
container), run the following command:

    docker run -P -ti --volumes-from BACKEND_CNTNR_NAME FRONTEND_IMG


Production Environnement
------------------------

### Setup

You should compile/install hardened kernel (with latest stable GrSec patch) on
each machine.

Prefer GNU/Linux distributions where most packages are compiled with `-fPIC`
and `-fstack-protector`, like Ubuntu or
[Gentoo Hardened](http://www.gentoo.org/proj/en/hardened/).

As machines aren't always in safe place (transportation, night before CTF,
...), disks should be encrypted.

**Always set strong password when it is possible** eg. SSL certificats, ...


#### Backend

##### Docker containers

Main Docker backend container relies on several other container:

* MySQL database;
* Database storage (as data only container);
* PKI storage;
* PKI shared storage;
* challenge files containers;
* the backend.

To have a fully working backend:

1. Create a data-only container:

```
docker run --name mysql_data -v /var/lib/mysql busybox
```

2. Setup the MySQL server:

```
docker run -d --name db_setup --volumes-from mysql_data -e MYSQL_ROOT_PASSWORD=mysecretpassword -e MYSQL_USER=fic -e MYSQL_PASSWORD=anotherpassword -e MYSQL_DATABASE=fic mysql mysqld --skip-name-resolve
```

Fill the database:

```
docker build -t db_filler db/
docker run --rm -it --link db_setup:db db_filler
```

Stop it:

```
docker stop db_setup
docker rm db_setup
```

3. Run the database container:

```
docker run -d --name db --volumes-from mysql_data -e MYSQL_USER=fic -e MYSQL_PASSWORD=anotherpassword -e MYSQL_DATABASE=fic mysql mysqld --skip-name-resolve
```

Here, we use the `--skip-name-resolve` option, because without internet connection, MySQL try to find hostname from IP and hang a lot.

4. Setup the PKI storages:

```
docker run --name pki_storage -v /var/www/fic-server/PKI busybox
docker run --name shared_storage -v /var/www/fic-server/shared busybox
```

5. Build the PKI configuration container:

```
docker build -t pki_setup pki/
```

6. Configure the PKI

For development purpose, you can run the default setup:

```
docker run --rm -it --volumes-from pki_storage --volumes-from shared_storage pki_setup
```

For production environment:

```
docker run --rm -it --volumes-from pki_storage --volumes-from shared_storage pki_setup IPorURL
```

Where `IPorURL` is the way the certificat will authenticate: if challengers
will access the frontend with a custom domain, indicate this domain
(eg. `epita_challenge.fic.local`); else indicate the IP of the **front** host
on the network (eg. `192.168.0.5`).

For example, 2015 PKI generation looks like:

```
docker run --rm -it --volumes-from pki_storage --volumes-from shared_storage pki_setup 192.168.0.5
```

7. Build the perl `Mcrypt` debian package

```
docker build -t perl-mcrypt perl-mcrypt/
docker run --name mcrypt_builder perl-mcrypt
docker cp mcrypt_builder:$(docker diff mcrypt_builder | grep -oE '[^ ]+deb$') ./
docker rm mcrypt_builder
```

8. Build and run the backend:

```
docker build -t backend .
docker run --rm -it --link db:db --volumes-from pki_storage --volumes-from shared_storage -v /home/files:/var/www/fic-server/files-in backend
```

Where `/home/files` is your local directory containing all challenge files.


##### Requirements

* `realpath`;
* `mysql`;
* `nginx` with `fastcgi` module;
* `php-fpm` with `mysql` module;
* `openssl` and `pwgen` for client certificat generation;
* `mcrypt`;
* `HTTP::Request::Common` perl module (provided by `libwww-perl`);
* `Digest::Whirlpool` perl module (provided by `lib-digest-whirlpool-perl`);
* `Mcrypt` from CPAN (`cpan -i Mcrypt`, on Debian, it requires `libltdl-dev` and
  `build-essential`) to decrypt submissions (see
  https://metacpan.org/pod/Mcrypt);

##### Files distribution

You need to manually place challenge given files in the tree. To avoid path
guessing, files path are hashed. To generate hashed paths, use the script
`gen_hash_link_files.sh`:

    mkdir $TO
    ./gen_hash_link_files.sh FROM TO

Where `FROM` is the directory with the orignal tree and `TO` the directory
where placed symlink.

##### Firewall rules

This machine shouldn't have any network connection, except outgoing one to the
frontend for synchronization.

##### Others setups

Indicate in `/etc/hosts.conf` IP(s) of the frontend.


### Run

Two scripts are available, depending if directories synchronization has to be
made or not.

You don't need to handle synchronization if it's done by a separate container
or if frontend is linked to backend.

The `launch.sh` and `launch_local.sh` scripts do all backend stuff for you:
synchronization with frontend (only `launch.sh`), submission checking and
smart static pages regeneration.


#### Frontend

Keep in mind that this is the machine exposed to participant.

##### Docker containers

1. Generate the synchronization SSH key on the backend:

```
sudo su -c "ssh-keygen -t rsa -b 8192 -N '' -f /var/www/fic_server/.ssh/id_rsa" synchro
```

2. Copy `~synchro/.ssh/id_rsa.pub` into `front_synchro/authorized_keys` file.

3. Run the `front_synchro` container:

```
docker build -t synchro front_synchro/
docker run -d --name fsync -p 2242:22 synchro
```

4. Run the backend.

A first synchronization have to be made before the next step. This
synchronization will copy the frontend private key and various required
certificates.

5. Run the frontend container:

```
docker build -t frontend front/
docker run --rm -it -p 80:80 -p 443:443 --volumes-from fsync frontend
```


##### Requirements

* `nginx` with those modules: `aio` (for fast delivery of huge
  content), `fastcgi`, `rewrite`, `ssl`;
* `php-fpm` with `mcrypt` module (for submission encryption);

##### Firewall rules

Expose to participants only 80 and 443 ports.

Expose on synchronization interface the 22 port, used for synchronization and
administration purpose from backend.

DROP **has to be** the default rule for INPUT, FORWARD and OUTPUT chains; use
CONNTRACK states.


### History

#### FIC2014

Two machines (DC7900: Core 2 Quad) were used : one for backend (Deimos) and one
for frontend (Phobos). They ran a GNU/Linux Gentoo Hardened with custom 3.2
kernel without module loading, unused and unecessary components and with all
GrSecurity features activated.

Each machine was two network interfaces: one was used to permit to the backend
machine to connect to the frontend (over IPv6). The second interface on the
backend was used for administration purpose (with a laptop not connected to
Internet). The second interface on the frontend was used to provide network
connectivity to participants.

Both frontend and backend were 2 500GB hard-drives with software RAID1. The
whole logical RAID disk was LUKS encrypted using Serpent algorithm.


The D Day
---------

### Interact with the scheduler

When you launch `launch.sh` or `launch_local.sh` script, a socket is open at
`/tmp/scheduler.sock`. Use `perl comm-socket.pl /tmp/scheduler.sock` to connect
to the scheduler. Consult `gen_site.pl` manual (`perldoc gen_site.pl`) for list
of available instructions.

### More

TODO