Skip to content

Local setup

This guide will help you set up everything you need for running xikolo and start developing right away.

Prerequisites

Repository

If you are part of the Xikolo core team at the HPI, you will need access to the project repository and GitLab.

  • You need at least read access for the xikolo group or direct (read) access to the web repository.
  • Ensure that you have created an SSH key pair (ed25519 is highly recommended), and add your public key to your GitLab profile.

Optional: To avoid having to enter your password each time you can add your key to your local machine using the following command.

ssh-add --apple-use-keychain ~/.ssh/id_ed25519

Software

Xikolo can be installed on Debian-based Linux or Mac computers. First, make sure the following software is installed on your machine:

Git: Version control

sudo apt install git

Will be installed automatically on first use.

RVM: Ruby version manager

Install RVM as described in

Ruby 3.4: The programming language for most of our services

rvm install 3.4.4

rvm install 3.4.4

Bundler: Ruby’s dependency manager

gem install bundler -v '~> 2.0'

Bun: JavaScript runtime and bundler for the frontend assets

Follow the installation instructions on the Bun website.

Install dependencies:

bun install

Note

--install-directory can be used to install the stubs in a specific directory, such as /usr/local/bin, or without sudo ~/.local/bin. This path must be in PATH.

RabbitMQ: Asynchronous messaging / event broadcasting between services

Tip

There is a RabbitMQ Web-UI (if you install it): http://localhost:15672

username: guest password: guest

sudo apt install rabbitmq-server

Optionally, install the Web UI:

sudo rabbitmq-plugins enable rabbitmq_management

brew install rabbitmq

To start as a service:

brew services start rabbitmq

Optionally, install the rabbitmq_management:

/usr/local/opt/rabbitmq/sbin/rabbitmq-plugins enable rabbitmq_management

PostgreSQL 17 (or later): A relational database

Note

The version used in production is still PostgreSQL 16. Potential differences need to be considered, e.g. when using a specific feature that is not available.

sudo apt install postgresql

brew install postgresql@17
brew link postgresql@17

To start as a service:

brew services start postgresql@17

Redis (7.0): A key-value store for caching and other use-cases

Note

Increase database number (databases option) to >8000 (e.g. 8192)

Install Redis from upstream archive to get a 7.0 version: https://redis.io/docs/getting-started/installation/install-redis-on-linux/.

Edit the file /etc/redis/redis.conf:

  • Replace databases 16 with databases 8192
  • Replace supervised no with supervised systemd

Finally, restart Redis for the changes to take effect:

sudo systemctl restart redis-server

brew install redis

To start as a service:

brew services start redis

Edit the file redis.conf:

nano $(brew --prefix)/etc/redis.conf
  • Replace databases 16 with databases 8192

Restart Redis:

brew services restart redis

Imagemagick 6: An image manipulation toolkit

sudo apt install imagemagick

brew install imagemagick@6

Minio: A local S3 server - needed for services that store files

The Minio server can then be started using:

minio server ~/xi/s3-data

You can also make it a user space service. Create these directories:

mkdir -p ~/.local/lib/minio
mkdir -p ~/.config/systemd/user

Add the file ~/.config/systemd/user/minio.service as follows:

[Unit]
Description=MinIO
Documentation=https://docs.min.io
Wants=network-online.target
After=network-online.target
AssertFileIsExecutable=/usr/local/bin/minio

[Service]
WorkingDirectory=/home/YOUR_USERNAME/.local/lib/minio
ExecStart=/usr/local/bin/minio server --address localhost:9000 /home/YOUR_USERNAME/.local/lib/minio
Restart=always
LimitNOFILE=65536
TimeoutStopSec=infinity
SendSIGKILL=no

[Install]
WantedBy=default.target

You can then use systemctl:

systemctl --user start minio
systemctl --user status minio
systemctl --user stop minio

brew install minio

To start as a service:

brew services start minio

Service runtime dependencies: The following system packages are needed to build or run our services

sudo apt install \
cmake \
ffmpeg \
gir1.2-gdkpixbuf-2.0 \
gir1.2-rsvg-2.0 \
libcurl4 \
libffi-dev \
libgirepository1.0-dev \
libgit2-dev \
libpq-dev \
librsvg2-dev \
libsodium23 \
libv8-dev \
libxml2-dev \
libxslt1-dev \
libz-dev \
pkg-config \
zip

brew install \
cmake \
libsodium

Setup

Configure the Postgres user

psql tries to connect to the Postgres server using the same username as your system user.

# For installing createuser
sudo apt install postgresql-client-common

sudo su - postgres
createuser -s -r your_system_username

First, check whether Postgres is running, e.g.

brew services

Install integration dependencies

This project can be used to set up all services, and run tests against all of them.

cd integration
bundle install

Install each service’s dependencies

bundle exec rake bundle:install

Carefully check the output for error messages (e.g. libraries still missing).

Configure S3

Run rake s3:configure in integration to set up all necessary buckets and configure their policies.

Make sure minio is running when executing this rake task. If your minio server is running at another endpoint or with different than the default credentials, adapt ~/.xikolo.development.yml accordingly.

Adopt the ~/.xikolo.development.yml (copy from xikolo/integration/features/support/lib/xikolo.development.yml) and try the minio default key and secret: set both to minioadmin.

Run rake s3:configure in integration to set up all needed buckets and configure their needed policies.

If the rake task rake s3:configure does not proceed, try the following:

  • brew services list shows that the config is loaded from something like /Users/<username>/Library/LaunchAgents/homebrew.mxcl.minio.plist. Inspecting this file reveals that it executes minio by running /usr/local/opt/minio/bin/minio server --config-dir=/usr/local/etc/minio --address=:9000 /usr/local/var/minio.
  • To get the secret and access key printed on the shell, shut down the service and run this command once manually. After you copied the keys to the Xikolo config file, you can use minio as a service again.

Set up databases and other backing services

This command will create databases and database tables as well as queues and exchanges in RabbitMQ and finally set up some example data for development purposes.

bundle exec rake reset

Install and compile frontend dependencies

This project uses Propshaft to manage frontend dependencies. Bun is used to bundle the JavaScript, the (legacy) CSS is compiled using Sass.

Everything is then delivered via Propshaft.

Running the following script starts both the Rails server and the asset bundler’s development server:

./bin/dev

Configure EditorConfig support

EditorConfig is a standard for some generic auto-formatting options supported by many IDEs.

Tip

Check https://editorconfig.org/ for your IDE’s built-in support. If not available, please install the corresponding add-on / editor plug-in.

Prevent push to master

Any push command which operates on the master branch will require confirmation.

Note

  1. Copy the snippet below to .git/hooks/pre-push
  2. Make the hook executable: chmod +x .git/hooks/pre-push
#!/bin/bash
protected_branch='master'
current_branch=$(git symbolic-ref HEAD | sed -e 's,.*/\(.*\),\1,')
if [ $protected_branch = $current_branch ]
then
    read -p "You're about to push master, is that what you intended? [y|n] " -n 1 -r < /dev/tty
    echo
    if echo $REPLY | grep -E '^[Yy]$' > /dev/null
    then
        exit 0 # push will execute
    fi
    exit 1 # push will not execute
else
    exit 0 # push will execute
fi

Taken from https://ghost.org/blog/prevent-master-push/.

Lint before commit

A pre-commit hook should be automatically installed through husky when running yarn install. The hook will run several formatting checks and auto-fixes before a commit gets created.

See lint-staged rules in package.json for applied commands.

Git commit message template

You can add a git commit message template to help you always have the right format.

Create a commit-template.txt file wherever you like.

Add this to the file:

# Use this format for the message header:
# <type>[optional scope]: <subject>
# optional scopes are bound to our services e.g.
# course, account, lanalytics and so on
#| <----- Type Maximum 50 Characters here -----> |

# The commit message body should contain the what and why, starting with a blank line
# For more information, see our guidelines: https://xikolo.pages.xikolo.de/web/development/workflows/review/#commits
#| <---- Try To Limit Each Line to a Maximum Of 70 Characters -----> |

Run the following command to set it up.

git config --global commit.template <filepath/commit-template.txt>

Set up your local dev tooling

There are multiple ways to work with your local copy of Xikolo. These tutorials provide hints for a few of them:

Starting the project

Overmind

Info

The Xikolo Rails needs supporting services to execute assets compilation and background jobs. You can use overmind with the command line in the project directory as follows:

See installation guide

brew install overmind

The rails app and supporting services will be started:

overmind start

Start the project with a different brand than default:

BRAND=your_brand overmind start -l course,web

Note

Make sure to also set the brand and site_name options to the brand in the config, e.g. in your xikolo.development.yml.

If you have done everything correctly and installed all the necessary tools, you should see the Xikolo platform at http://localhost:3000.

External Services with Docker

All necessary external services (PostgreSQL RabbitMQ, Redis, Minio) can be provisioned via Docker using the included docker-compose.yml. Initialize and start the defined containers:

docker-compose up

The database URL for Xikolo apps must then be set as an environment variable before executing any utility or application with database access, e.g.

export DATABASE_URL=postgresql://localhost:5432
bundle exec rake db:prepare
bundle exec rails s

To run overmind with the docker-based databases, load the specific .env.docker file:

OVERMIND_ENV=.env.docker overmind start

For many IDEs providing run configurations, you can set the environment variable in the IDE settings or the run configurations, e.g. in RubyMine.

Locales

The project uses i18n for internationalization (i.e., localization of content), making these locales accessible to the frontend application using the i18n-js gem. This gem exports the locales into JSON format and stores them in tmp/cache/xikolo/i18n for each brand (configured in ./config/i18n). These JSON objects are imported in the Webpack code, then, and consumed by the i18n-js npm package.

To ensure that the locales are available when Webpack is built and to avoid potential build failures, execute the gem’s CLI command (i18n export) before executing yarn run build. Alternatively, you can simply use make assets, which automates all the required steps.

Seeds

The rake reset command that is part of the installation instructions also “seeds” your databases, creating lots of example data (such as courses, course content, and user accounts).

You can run always run again bundle exec rake reset in the integration folder of your local working environment to put your local copy in a blank, clean state - which can be useful when starting a new task, or trying to reproduce a bug.

Based on the seeds, there will be the following users (besides other less important seed users):

  • Regular user Email: kevin.cool@example.com Password: qwe123qwe

  • Administrator Email: admin@example.com Password: administrator

  • Teacher Email: tom@example.com Password: teaching

Testing

To run the test suite and make sure you have the necessary drivers, please see the full ‘how-to’ guide regarding running tests.

Troubleshooting

  • Apple Silicon: If you are using a MacBook with an Apple Silicon chip (instead of Intel) be sure to read this article first.
  • Known issues: There is a list of possible problems with solutions for Mac OS X

Update this Documentation

If discrepancies are noticed when carrying your local setup, please update this documentation.

  • The documentation can be found in the repository here: docs/app/development/local_setup/index.md

To view and edit this documentation locally, you need to install uv (if not already installed). The following commands are to be run from the top-level project directory:

sudo apt install pipx
pipx install uv

brew install uv

To run the docs:

uv run mkdocs serve