Submission server/infrastructure for the SRS challenge at FIC https://fic.srs.epita.fr/
Go to file
nemunaire 28815f68c2
continuous-integration/drone/push Build is failing Details
repochecker: Remove indication on how to circumvent forbidden-string
2024-05-18 03:21:46 +02:00
.gitlab-ci CI: Add remote-challenge-sync-airbus 2024-04-19 11:48:59 +02:00
admin sync: Allow justified as flag type 2024-05-18 00:28:59 +02:00
checker checker: Refactor + ensure theme is not disabled 2024-04-19 11:48:59 +02:00
configs confs: Add _ as an expected theme char 2024-04-19 11:49:00 +02:00
dashboard Remove unused StripPrefix 2024-04-19 11:48:58 +02:00
doc README: Add diagrams 2024-04-19 11:48:56 +02:00
evdist evdist: Publish next settings change 2024-04-19 11:48:59 +02:00
fickit-pkg fickit-pkg/mdadm: Don't fail if /etc/init.d/ is empty 2024-04-19 11:48:58 +02:00
frontend/fic ui: Prepare publication 2024-05-16 13:09:13 +02:00
generator Split Unlock standalone exercices between themes and standalone ex 2024-04-19 11:48:59 +02:00
libfic checker: Ensure a flag is found before considering a good response 2024-04-19 11:49:00 +02:00
nixos Split backend service into checker and generator 2024-04-19 11:48:55 +02:00
password_paper password_paper: Update 2022-05-31 16:42:17 +02:00
qa chore(deps): lock file maintenance 2024-04-19 11:49:00 +02:00
receiver Allow more parameters to be passed in environment 2024-04-19 11:48:58 +02:00
remote challenge-sync-airbus: Avoid concurrent map write 2024-04-19 11:49:00 +02:00
repochecker repochecker: Remove indication on how to circumvent forbidden-string 2024-05-18 03:21:46 +02:00
settings evdist: Publish next settings change 2024-04-19 11:48:59 +02:00
udev udev: ready to FIC2019 2019-01-22 08:49:44 +01:00
.dockerignore Split backend service into checker and generator 2024-04-19 11:48:55 +02:00
.drone-manifest-fic-admin.yml CI: create Docker manifest image 2021-02-23 09:11:47 +01:00
.drone-manifest-fic-checker.yml Split backend service into checker and generator 2024-04-19 11:48:55 +02:00
.drone-manifest-fic-dashboard.yml CI: create Docker manifest image 2021-02-23 09:11:47 +01:00
.drone-manifest-fic-evdist.yml evdist: New project to handle settings programming 2022-06-08 02:57:29 +02:00
.drone-manifest-fic-frontend-ui.yml CI: Also push manifest of fic-frontend-ui 2024-04-19 11:49:00 +02:00
.drone-manifest-fic-generator.yml Split backend service into checker and generator 2024-04-19 11:48:55 +02:00
.drone-manifest-fic-get-remote-files.yml Introducing get-remote-files to download remote files 2024-04-19 11:48:58 +02:00
.drone-manifest-fic-nginx.yml Introduce fic-nginx 2021-07-21 03:03:25 +02:00
.drone-manifest-fic-qa.yml CI: create Docker manifest image 2021-02-23 09:11:47 +01:00
.drone-manifest-fic-receiver.yml Rename frontend as receiver 2024-04-19 11:48:51 +02:00
.drone-manifest-fic-repochecker.yml CI: add Dockerfile for repochecker (used for student's CI) 2021-05-14 00:44:55 +02:00
.drone-manifest-fickit-deploy.yml Introduce fickit-deploy image 2024-04-19 11:48:56 +02:00
.drone.yml CI: Also push manifest of fic-frontend-ui 2024-04-19 11:49:00 +02:00
.gitignore Introducing get-remote-files to download remote files 2024-04-19 11:48:58 +02:00
.gitlab-ci.yml CI: Add remote-challenge-sync-airbus 2024-04-19 11:48:59 +02:00
Dockerfile-admin chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-checker chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-dashboard ui: Remove old content 2024-05-16 13:09:13 +02:00
Dockerfile-deploy chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-evdist chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-frontend-ui frontend: migration to SvelteKit 2 + Sveltestrap 6 2024-04-19 11:48:58 +02:00
Dockerfile-generator chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-get-remote-files chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-nginx frontend: migration to SvelteKit 2 + Sveltestrap 6 2024-04-19 11:48:58 +02:00
Dockerfile-qa chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-receiver chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-remote-challenge-sync-airbus chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-remote-scores-sync-zqds chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
Dockerfile-repochecker chore(deps): update alpine docker tag to v3.19 2024-04-19 11:48:58 +02:00
LICENSE Add LICENSE: chose MIT 2018-05-11 15:03:11 +02:00
README.md README: Add diagrams 2024-04-19 11:48:56 +02:00
default.nix Add Nix flakes 2022-05-31 00:09:27 +02:00
docker-compose.yml fickit: Share the SYNC directory 2024-04-19 11:48:57 +02:00
entrypoint-receiver.sh chbase: Use same strategy for relative paths as qa 2024-04-19 11:48:55 +02:00
fickit-backend.yml fickit: New script to upgrade backend without reboot 2024-04-19 11:49:00 +02:00
fickit-boot.yml chore(deps): update linuxkit/kernel docker tag to v6 2024-04-19 11:48:58 +02:00
fickit-frontend.yml fickit: Fix launch of challenge-sync containers 2024-04-19 11:49:00 +02:00
fickit-prepare.yml fickit: Update linuxkit images 2024-04-19 11:49:00 +02:00
fickit-update.yml fickit: Update linuxkit images 2024-04-19 11:49:00 +02:00
flake.lock Add Nix flakes 2022-05-31 00:09:27 +02:00
flake.nix Split backend service into checker and generator 2024-04-19 11:48:55 +02:00
go.mod chore(deps): update module golang.org/x/oauth2 to v0.19.0 2024-04-19 11:49:00 +02:00
go.sum chore(deps): update module golang.org/x/oauth2 to v0.19.0 2024-04-19 11:49:00 +02:00
htdocs-admin admin: Embed static assets into binary 2021-06-09 23:26:40 +02:00
htdocs-dashboard dashboard: came back online 2018-12-08 20:35:36 +01:00
qa-add-todo.sh qa: Update scripts 2024-04-19 11:48:56 +02:00
qa-common.sh qa: Update scripts 2024-04-19 11:48:56 +02:00
qa-fill-todo.sh qa: Update scripts 2024-04-19 11:48:56 +02:00
qa-fill-view-overview.sh qa: Update scripts 2024-04-19 11:48:56 +02:00
qa-fill-view.sh qa: Update scripts 2024-04-19 11:48:56 +02:00
qa-merge-db.sh qa: Update scripts 2024-04-19 11:48:56 +02:00
renovate.json renovate: Fix regexp for linuxkit files 2024-04-19 11:48:56 +02:00
shell.nix Add Nix flakes 2022-05-31 00:09:27 +02:00
update-deps.sh deps: Update vendorSha256 with script. 2022-08-24 00:51:49 +02:00

README.md

FIC Forensic CTF Platform

This is a CTF server for distributing and validating challenges. It is design to be robust, so it uses some uncommon technics like client certificate for authentication, lots of state of the art cryptographic methods and aims to be deployed in a DMZ network architecture.

This is a monorepo, containing several micro-services :

  • admin is the web interface and API used to control the challenge and doing synchronization.
  • checker is an inotify reacting service that handles submissions checking.
  • dashboard is a public interface to explain and follow the conquest, aims to animate the challenge for visitors.
  • evdist is an inotify reacting service that handles settings changes during the challenge (eg. a 30 minutes event where hints are free, ...).
  • generator takes care of global and team's files generation.
  • qa is an interface dedicated to challenge development, it stores reports to be treated by challenges creators.
  • receiver is only responsible for receiving submissions. It is the only dynamic part accessibe to players, so it's codebase is reduce to the minimum. It does not parse or try to understand players submissions, it just write it down to a file in the file system. Parsing and treatment is made by the checker.
  • remote/challenge-sync-airbus is an inotify reacting service that allows us to synchronize scores and exercice validations with the Airbus scoring platform.
  • remote/scores-sync-zqds is an inotify reacting service that allows us to synchronize scores with the ZQDS scoring platform.
  • repochecker is a side project to check offline for synchronization issues.

Here is how thoses services speak to each others:

Overview of the micro-services

In the production setup, each micro-service runs in a dedicated container, isolated from each other. Moreover, two physical machines should be used:

  • phobos communicates with players: displaying the web interface, authenticate teams and players, storing contest files and handling submissions retrieval without understanding them. It can't access deimos so its job stops after writing requests on the filesystem.
  • deimos is hidden from players, isolated from the network. It can only access phobos via a restricted ssh connection, to retrieve requests from phobos filesystem and pushing to it newly generated static files.

Concretely, the L2 looks like this:

Layer 2 connections

So, the general filesystem is organized this way:

  • DASHBOARD contains files structuring the content of the dashboard screen(s).
  • FILES stores the contest file to be downloaded by players. To be accessible without authentication and to avoid bruteforce, each file is placed into a directory with a hashed name (the original file name is preserved). It's rsynced as is to deimos.
  • GENERATOR contains a socket to allow other services to communicate with the generator.
  • PKI takes care of the PKI used for the client certiciate authorization process, and more generaly, all authentication related files (htpasswd, dexidp config, ...). Only the shared subdirectory is shared with deimos, private key and teams P12 don't go out.
  • SETTINGS stores the challenge config as wanted by admins. It's not always the config in use: it uses can be delayed waiting for a trigger.
  • SETTINGSDIST is the challenge configuration in use. It is the one shared with players.
  • startingblock keep the started state of the challenge. This helps nginx to know when it can start distributing exercices related files.
  • TEAMS stores the static files generated by the generator, there is one subdirectory by team (id of the team), plus some files at the root, which are common to all teams. There is also symlink pointing to team directory, each symlink represent an authentication association (certificate ID, OpenID username, htpasswd user, ...).
  • submissions is the directory where the receiver writes requests. It creates subdirectories at the name of the authentication association, as seen in TEAMS, checker then resolve the association regarding TEAMS directory. There is also a special directory to handle team registration.

Here is a diagram showing how each micro-service uses directories it has access to (blue for read access, red for write access):

Usage of directories by each micro-service

Local developer setup

Using Docker

Use docker-compose build, then docker-compose up to launch the infrastructure.

After booting, you'll be able to reach the main interface at: http://localhost:8042/ and the admin one at: http://localhost:8081/ (or at http://localhost:8042/admin/). The dashboard is available at http://localhost:8042/dashboard/ and the QA service at http://localhost:8042/qa/.

In this setup, there is no authentication. You are identfied as a team. On first use you'll need to register.

Import folder

Local import folder

The following changes is only required if your are trying to change the local import folder ~/fic location.

Make the following changes inside this file docker-compose.yml:

23    volumes:
24 -     - ~/fic:/mnt/fic:ro
24 +     - <custom-path-to-import-folder>/fic:/mnt/fic:ro
Git import

A git repository can be used:

29 -    command: --baseurl /admin/ -localimport /mnt/fic -localimportsymlink
29 +    command: --baseurl /admin/ -localimport /mnt/fic -localimportsymlink -git-import-remote git@gitlab.cri.epita.fr:ing/majeures/srs/fic/2042/challenges.git
Owncloud import folder

If your are trying to use the folder available with the Owncloud service, make the following changes inside this file docker-compose.yml:

29 -    command: --baseurl /admin/ -localimport /mnt/fic -localimportsymlink
29 +    command: --baseurl /admin/ -clouddav=https://owncloud.srs.epita.fr/remote.php/webdav/FIC%202019/ -clouduser <login_x> -cloudpass '<passwd>'

Manual builds

Running this project requires a web server (configuration is given for nginx), a database (currently supporting only MySQL/MariaDB), a Go compiler for the revision 1.18 at least and a inotify-aware system. You'll also need NodeJS to compile some user interfaces.

  1. Above all, you need to build Node projects:

     cd frontend/fic; npm install && npm run build
     cd qa/ui; npm install && npm run build
    
  2. First, you'll need to retrieve the dependencies:

     go mod vendor
    
  3. Then, build the three Go projects:

     go build -o fic-admin ./admin
     go build -o fic-checker ./checker
     go build -o fic-dashboard ./dashboard
     go build -o fic-generator ./generator
     go build -o fic-qa ./qa
     go build -o fic-receiver ./receiver
     go build -o fic-repochecker ./repochecker
     ...
    
  4. Before launching anything, you need to create a database:

     mysql -u root -p <<EOF
     CREATE DATABASE fic;
     CREATE USER fic@localhost IDENTIFIED BY 'fic';
     GRANT ALL ON fic.* TO fic@localhost;
     EOF
    

    By default, expected credentials for development purpose is fic, for both username, password and database name. If you want to use other credentials, define the corresponding environment variable: MYSQL_HOST, MYSQL_USER, MYSQL_PASSWORD and MYSQL_DATABASE. Those variables are the one used by the mysql docker image, so just link them together if you use containers.

  5. Launch it!

     ./fic-admin &
    

    After initializing the database, the server will listen on http://localhost:8081/: this is the administration part.

     ./fic-generator &
    

    This daemon generates static and team related files and then waits another process to tell it to regenerate some files.

     ./fic-receiver &
    

    This one exposes an API that gives time synchronization to clients and handle submission reception (but without treating them).

     ./fic-checker &
    

    This service waits for new submissions (expected in submissions directory). It only watchs modifications on the file system, it has no web interface.

     ./fic-dashboard &
    

    This last server runs the public dashboard. It serves all file, without the need of a webserver. It listens on port 8082 by default.

     ./fic-qa &
    

    If you need it, this will launch a web interface on the port 8083 by default, to perform quality control.

For the moment, a web server is mandatory to serve static files, look at the samples given in the configs/nginx directory. You need to pick one base configation flavor in the configs/nginx/base directory, and associated with an authentication mechanism in configs/nginx/auth (named the file fic-auth.conf in /etc/nginx), and also pick the corresponding configs/nginx/get-team file, you named fic-get-team.conf.