3.0.0a1 (CTFd#1523)

Alpha release of CTFd v3. 

# 3.0.0a1 / 2020-07-01

**General**

- CTFd is now Python 3 only
- Render markdown with the CommonMark spec provided by `cmarkgfm`
- Render markdown stripped of any malicious JavaScript or HTML.
  - This is a significant change from previous versions of CTFd where any HTML content from an admin was considered safe.
- Inject `Config`, `User`, `Team`, `Session`, and `Plugin` globals into Jinja
- User sessions no longer store any user-specific attributes.
  - Sessions only store the user's ID, CSRF nonce, and an hmac of the user's password
  - This allows for session invalidation on password changes
- The user facing side of CTFd now has user and team searching
- GeoIP support now available for converting IP addresses to guessed countries

**Admin Panel**

- Use EasyMDE as an improved description/text editor for Markdown enabled fields.
- Media Library button now integrated into EasyMDE enabled fields
- VueJS now used as the underlying implementation for the Media Library
- Fix setting theme color in Admin Panel
- Green outline border has been removed from the Admin Panel

**API**

- Significant overhauls in API documentation provided by Swagger UI and Swagger json
- Make almost all API endpoints provide filtering and searching capabilities
- Change `GET /api/v1/config/<config_key>` to return structured data according to ConfigSchema

**Themes**

- Themes now have access to the `Configs` global which provides wrapped access to `get_config`.
  - For example, `{{ Configs.ctf_name }}` instead of `get_ctf_name()` or `get_config('ctf_name')`
- Themes must now specify a `challenge.html` which control how a challenge should look.
- The main library for charts has been changed from Plotly to Apache ECharts.
- Forms have been moved into wtforms for easier form rendering inside of Jinja.
  - From Jinja you can access forms via the Forms global i.e. `{{ Forms }}`
  - This allows theme developers to more easily re-use a form without having to copy-paste HTML.
- Themes can now provide a theme settings JSON blob which can be injected into the theme with `{{ Configs.theme_settings }}`
- Core theme now includes the challenge ID in location hash identifiers to always refer the right challenge despite duplicate names

**Plugins**

- Challenge plugins have changed in structure to better allow integration with themes and prevent obtrusive Javascript/XSS.
  - Challenge rendering now uses `challenge.html` from the provided theme.
  - Accessing the challenge view content is now provided by `/api/v1/challenges/<challenge_id>` in the `view` section. This allows for HTML to be properly sanitized and rendered by the server allowing CTFd to remove client side Jinja rendering.
  - `challenge.html` now specifies what's required and what's rendered by the theme. This allows the challenge plugin to avoid having to deal with aspects of the challenge besides the description and input.
  - A more complete migration guide will be provided when CTFd v3 leaves beta
- Display current attempt count in challenge view when max attempts is enabled
- `get_standings()`, `get_team_stanadings()`, `get_user_standings()` now has a fields keyword argument that allows for specificying additional fields that SQLAlchemy should return when building the response set.
  - Useful for gathering additional data when building scoreboard pages
- Flags can now control the message that is shown to the user by raising `FlagException`
- Fix `override_template()` functionality

**Deployment**

- Enable SQLAlchemy's `pool_pre_ping` by default to reduce the likelihood of database connection issues
- Mailgun email settings are now deprecated. Admins should move to SMTP email settings instead.
- Postgres is now considered a second class citizen in CTFd. It is tested against but not a main database backend. If you use Postgres, you are entirely on your own with regards to supporting CTFd.
- Docker image now uses Debian instead of Alpine. See CTFd#1215 for rationale.
- `docker-compose.yml` now uses a non-root user to connect to MySQL/MariaDB
- `config.py` should no longer be editting for configuration, instead edit `config.ini` or the environment variables in `docker-compose.yml`

.eslintrc.js

@@ -13,5 +13,6 @@ module.exports = {
         "sourceType": "module"
     },
     "rules": {
+        "no-unused-vars": ["error", { "argsIgnorePattern": "^_" }]
     }
 };

.github/ISSUE_TEMPLATE.md

@@ -6,9 +6,9 @@ If this is a feature request please describe the behavior that you'd like to see
 
 **Environment**:
 
-  - CTFd Version/Commit:
-  - Operating System:
-  - Web Browser and Version:
+- CTFd Version/Commit:
+- Operating System:
+- Web Browser and Version:
 
 **What happened?**
 
@@ -17,4 +17,3 @@ If this is a feature request please describe the behavior that you'd like to see
 **How to reproduce your issue**
 
 **Any associated stack traces or error logs**
-

.github/workflows/lint.yml

@@ -0,0 +1,43 @@
+---
+name: Linting
+
+on: [push]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        python-version: ['3.6']
+        TESTING_DATABASE_URL: ['sqlite://']
+
+    name: Linting
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+          architecture: x64
+      - name: Install dependencies
+        run: |
+            python -m pip install --upgrade pip
+            python -m pip install -r development.txt
+            sudo yarn install --non-interactive
+            sudo yarn global add [email protected]
+
+      - name: Lint
+        run: make lint
+
+      - name: Lint Dockerfile
+        uses: brpaz/hadolint-action@master
+        with:
+          dockerfile: "Dockerfile"
+
+      - name: Lint docker-compose
+        run: |
+            python -m pip install docker-compose==1.26.0
+            docker-compose -f docker-compose.yml config
+

.github/workflows/mysql.yml

@@ -0,0 +1,49 @@
+---
+name: CTFd MySQL CI
+
+on: [pull_request]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    services:
+      mysql:
+        image: mysql
+        ports:
+          - 3306:3306
+      redis:
+        image: redis
+        ports:
+          - 6379:6379
+
+    strategy:
+      matrix:
+        python-version: ['3.6']
+        TESTING_DATABASE_URL: ['mysql+pymysql://root@localhost/ctfd']
+
+    name: Python ${{ matrix.python-version }}
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+          architecture: x64
+      - name: Install dependencies
+        run: |
+            python -m pip install --upgrade pip
+            python -m pip install -r development.txt
+            sudo yarn install --non-interactive
+
+      - name: Test
+        run: |
+            sudo rm -f /etc/boto.cfg
+            make test
+        env:
+            AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
+            AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
+
+      - name: Coverage
+        run: codecov
+

.github/workflows/postgres.yml

@@ -0,0 +1,58 @@
+---
+name: CTFd Postgres CI
+
+on: [pull_request]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+    services:
+      postgres:
+        image: postgres
+        ports:
+          - 5432:5432
+        env:
+          POSTGRES_HOST_AUTH_METHOD: trust
+          POSTGRES_DB: ctfd
+        # Set health checks to wait until postgres has started
+        options: >-
+          --health-cmd pg_isready
+          --health-interval 10s
+          --health-timeout 5s
+          --health-retries 5
+      redis:
+        image: redis
+        ports:
+          - 6379:6379
+
+    strategy:
+      matrix:
+        python-version: ['3.6']
+        TESTING_DATABASE_URL: ['postgres://postgres@localhost/ctfd']
+
+    name: Python ${{ matrix.python-version }}
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+          architecture: x64
+      - name: Install dependencies
+        run: |
+            python -m pip install --upgrade pip
+            python -m pip install -r development.txt
+            sudo yarn install --non-interactive
+
+      - name: Test
+        run: |
+            sudo rm -f /etc/boto.cfg
+            make test
+        env:
+            AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
+            AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
+
+      - name: Coverage
+        run: codecov
+

.github/workflows/sqlite.yml

@@ -0,0 +1,41 @@
+---
+name: CTFd SQLite CI
+
+on: [pull_request]
+
+jobs:
+  build:
+
+    runs-on: ubuntu-latest
+
+    strategy:
+      matrix:
+        python-version: ['3.6']
+        TESTING_DATABASE_URL: ['sqlite://']
+
+    name: Python ${{ matrix.python-version }}
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup python
+        uses: actions/setup-python@v2
+        with:
+          python-version: ${{ matrix.python-version }}
+          architecture: x64
+      - name: Install dependencies
+        run: |
+            python -m pip install --upgrade pip
+            python -m pip install -r development.txt
+            sudo yarn install --non-interactive
+            sudo yarn global add [email protected]
+
+      - name: Test
+        run: |
+            sudo rm -f /etc/boto.cfg
+            make test
+        env:
+            AWS_ACCESS_KEY_ID: AKIAIOSFODNN7EXAMPLE
+            AWS_SECRET_ACCESS_KEY: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
+
+      - name: Coverage
+        run: codecov
+

.gitignore

@@ -36,6 +36,7 @@ pip-delete-this-directory.txt
 htmlcov/
 .tox/
 .coverage
+.coverage.*
 .cache
 nosetests.xml
 coverage.xml
@@ -57,6 +58,7 @@ target/
 
 *.db
 *.log
+*.log.*
 .idea/
 .vscode/
 CTFd/static/uploads

.isort.cfg

@@ -0,0 +1,7 @@
+[settings]
+multi_line_output=3
+include_trailing_comma=True
+force_grid_wrap=0
+use_parentheses=True
+line_length=88
+skip=migrations

.travis.yml

@@ -1,11 +1,10 @@
 language: python
-dist: xenial
+dist: bionic
 cache:
     - pip
     - yarn
 services:
     - mysql
-    - postgresql
     - redis-server
 addons:
   apt:
@@ -17,21 +16,16 @@ addons:
 env:
     - TESTING_DATABASE_URL='mysql+pymysql://root@localhost/ctfd'
     - TESTING_DATABASE_URL='sqlite://'
-    - TESTING_DATABASE_URL='postgres://postgres@localhost/ctfd'
 python:
-    - 2.7
     - 3.6
 before_install:
     - sudo rm -f /etc/boto.cfg
     - export AWS_ACCESS_KEY_ID=AKIAIOSFODNN7EXAMPLE
     - export AWS_SECRET_ACCESS_KEY=wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
-    - python3.6 -m pip install black==19.3b0
 install:
     - pip install -r development.txt
     - yarn install --non-interactive
     - yarn global add [email protected]
-before_script:
-    - psql -c 'create database ctfd;' -U postgres
 script:
     - make lint
     - make test