A Korean comic viewer that provides automatic dictionary lookups / translations.

Demo: https://reader.velchees.dev/

Features:

• Text is OCR'd using docTR (custom trained using a synthetic dataset).
• Dictionary data is gathered from Wiktionary and the National Institute of Korean Language's dictionary.
• Machine translations are generated via an LLM. LLMs are also used as a context-aware way of scoring dictionary definitions.

Screenshots

The instructions below assume you're using Ubuntu. Other OSes are untested but the only differences should come from how the dependencies are installed.

Installing via Docker is recommended as it prevents access to anything outside of the app folder. (This should also be the case without Docker, but mistakes happen.)

There is no login system, so if you plan to make the app accessible from the internet, it's recommended that you either

• make it only accessible from a WireGuard connection
• or add a login system via NGINX's Basic Auth module

Install the following:

• Docker
• git
• Python

(optional) If you have a NVIDIA GPU and want to use it for inference, run:

distribution=$(. /etc/os-release;echo $ID$VERSION_ID)
curl -s -L https://nvidia.github.io/nvidia-docker/gpgkey | apt-key add -
curl -s -L https://nvidia.github.io/nvidia-docker/$distribution/nvidia-docker.list | tee /etc/apt/sources.list.d/nvidia-docker.list

apt-get update && apt-get install -y nvidia-container-toolkit
systemctl restart docker

Clone the repo:

git clone git@github.com:LiteralGenie/reader.git
cd reader

Create a config file by copying the default template.
See the Config section for details. Note that GPU-acceleration is disabled by default.

Downloading custom weights for det_weights and reco_weights from here is also heavily recommended.

cp config_example.toml config.toml

Build the dictionary

cd ./core
python3 -m venv venv
. ./venv/bin/activate
pip install -r requirements.txt
python src/scripts/build_dictionary.py
cd ..

Build and run the Docker image

# Build with GPU
docker compose build --build-arg cuda="true"

# Or build without GPU
docker compose build

# Run
docker compose up -d

The app should then be running at http://localhost:9494

Install the following:

• Docker
• Python
• Node.js
• CUDA - optional and requries a NVIDIA GPU, but it's a huge speed-up

Clone the repo:

git clone git@github.com:LiteralGenie/reader.git
cd reader

Create a config file by copying the default template. Set api_host = localhost in the config.

Downloading custom weights for det_weights and reco_weights from here is also heavily recommended.

See the Config section for more details. Note that GPU-acceleration is disabled by default.

cp config_example.toml config.toml

Install dependencies:

cd ./core
python3 -m venv venv
. ./venv/bin/activate
pip install -r requirements.txt

cd ../web
npm install

cd ..

Build the dictionary

cd ./core
. ./venv/bin/activate
python src/scripts/build_dictionary.py
cd ..

(optional) If you plan to enable GPU-acceleration (which requires a NVIDIA GPU), you'll also need to recompile certain dependencies:

CMAKE_ARGS="-DGGML_CUDA=on" pip install --upgrade --force-reinstall --no-cache-dir python-doctr llama-cpp-python

And finally launch the API and web servers. Each needs to be launched in a separate terminal (or screen)

cd core
. ./venv/bin/activate
python src/run_server.py

cd web
npm run build
HOST=0.0.0.0 PORT=3030 BODY_SIZE_LIMIT=500M node build

Some notable config options / defaults are...

• Enable GPU-acceleration for the OCR step by setting use_gpu_for_ocr = true
• Enable GPU-acceleration for machine translations / definition sorting by setting llm_num_gpu_layers to a positive number.
• Definition sorting is also disabled by default (because it's super slow on CPU) but can be enabled by setting use_llm_for_definition_sort = true
• Custom weights for the OCR models can be downloaded from https://github.com/LiteralGenie/comic_ocr/releases and enabled by modifying det_weights and reco_weights.
• If using Docker, set api_host = core (the default). Otherwise set api_host = localhost.

If using Docker, do NOT modify the root_image_folder option. Also, the paths specified in det_weights and reco_weights (if any) should point to somewhere in the data folder.

For example:

# Path to model weights (*.pt). Leave blank ("") to use default weights (not recommended)
det_weights = "data/models/db_resnet50.pt"
reco_weights = "data/models/parseq.pt"

This is because only the data folder is mounted to the Docker container, any other paths will not be visible to the container.

Series and chapters can be added through the web gui (via file upload or URL), but for bulk imports, copying the files to the root_image_folder specified in config.toml (default reader/data/series) may be faster.

It's a known issue that things will occasionally get stuck (like chapter imports). Restarting the backend should fix this:

# If using Docker, find the container name
docker container ls
# And replace reader-core-1 in the below with it
docker container restart reader-core-1

###

# If not using Docker, kill the process (eg ctrl+c) and rerun the launch command
python src/run_server.py

• New OCR blocks cannot be added through the web GUI at the moment.
  • If you really need it, you'll have you edit the _reader_data.sqlite database in the folder for that chapter.
  • As a workaround you could just edit a nearby block to include the missing text.
  • The boxes tend to be correct 99% of the time so this has been low-priority.
• Only the series title / cover and the chapter title are currently displayed / saved. I do not plan to support other metadata like genre tags and series descriptions.