DynBench Frontend

DynBench is a web application for generating robust benchmark records by transforming question-query pairs using AI models.

---

Table of Contents

• DynBench Frontend
  • Features
  • Prerequisites
  • Installation
    • Clone the repository
    • Install dependencies
  • Configuration
    • Environment Variables
  • Running the Application
    • Running with Python (Development)
    • Running with Docker
    • Running with Docker Compose (optional)
  • Usage
  • Benchmark Datasets
  • Project Structure
  • Contributing
  • License
  • Appendix
    • List of languages by number of speakers in Europe

---

Features

• Generate new benchmark items from question-query pairs (natural-language questions with SPARQL queries)

• Support for generating new questions-query pairs from questions written in multiple languages (20 languages)

• Adjustable comparability of generated questions through the use of entities for generated questions (importance of entities in generated questions in comparison to is chosen to ensure difficulty to be "easy", "similar", "hard", or "random")

• Provide random, multilingual samples from benchmark datasets for easier tool exploration

• Feedback collection for computed transformation steps

Prerequisites

• Python 3.10+

• Docker (optional, for containerized deployment)

Installation

Clone the repository

git clone <repository-url>
cd DynBench-Frontend

Install dependencies

Using pip:

pip install -r requirements.txt

Or using pipenv:

pipenv install

Configuration

The application requires environment variables to be set. Create a .env file in the project root:

DYNBENCH=<Path to LLM transform endpoint>
MODEL=<Your model>

Environment Variables

Variable	Description	Default
DYNBENCH	URL of the DynBench backend service	-
MODEL	AI model to use for transformations	-
GITHUB_REPO	GitHub repository URL for the fork ribbon	https://github.com/WSE-research/DynBench-Frontend.git

Running the Application

Running with Python (Development)

1. Ensure you have installed the dependencies:

   pip install -r requirements.txt

2. Set up your .env file (see Configuration)

3. Run the Streamlit application:

   streamlit run app.py

4. The application will open in your browser at http://localhost:8501

Running with Docker

1. Build the Docker image:

   docker build -t dynbench-frontend .

2. Run the container with environment variables:

   docker run -p 8501:8501 \
          -e DYNBENCH=http://your-backend-url:port/transform \
          -e MODEL=mistral-small \
          dynbench-frontend

3. The application will be available at http://localhost:8501

Running with Docker Compose (optional)

Create a docker-compose.yml:

version: '3.8'
services:
  dynbench-frontend:
    build: .
    ports:
      - "8501:8501"
    environment:
      - DYNBENCH=http://your-backend-url:port/transform
      - MODEL=mistral-small
    restart: unless-stopped

Then run:

docker-compose up -d

Usage

1. Select difficulty: Choose from easy, normal, hard, or random

2. Select language: Pick from 20 supported languages

3. Enter question and query: Provide a natural language question and its corresponding SPARQL query

4. Generate: Click the "Generate" button to transform the pair

5. Review results: Check the transformed question and query

6. Provide feedback: Use the OK/Wrong buttons to give feedback on each result

Benchmark Datasets

The application includes sample datasets: - benchmarks/DynQALD.json - Question Answering Benchmark - benchmarks/DynRuBQ.json - Russian/English Benchmark

Project Structure

DynBench-Frontend/
├── app.py           # Main Streamlit application
├── requirements.txt    # Python dependencies
├── Dockerfile          # Docker configuration
├── .env               # Environment variables
├── css/               # Stylesheets
├── images/            # Application images
├── js/                # JavaScript files
└── benchmarks/        # Benchmark data

Contributing

1. Fork the repository

2. Create a feature branch

3. Commit your changes

4. Push to the branch

5. Open a Pull Request

License

This project is licensed under the MIT License.

Appendix

List of languages by number of speakers in Europe

(top 20 according to Wikipedia)

Position	Language	Code	Native speakers	Total speakers
1	Russian	ru	106,000,000	160,000,000
2	German	de	85,000,000	130,000,000
3	French	fr	67,000,000	112,000,000
4	English	en	67,000,000	280,000,000
5	Italian	it	58,000,000	72,000,000
6	Spanish	es	40,000,000	76,000,000
7	Polish	pl	38,000,000
8	Ukrainian	uk	32,600,000
9	Romanian	ro	24,000,000	28,000,000
10	Dutch	nl	22,000,000	24,000,000
11	Turkish	tr	15,752,673
12	Bavarian	bar	14,000,000
13	Portuguese	pt	11,000,000
14	Hungarian	hu	11,000,000
15	Greek	el	11,000,000
16	Czech	cs	10,600,000
17	Swedish	sv	10,000,000	11,000,000
18	Catalan	ca	10,000,000
19	Serbian	sr	9,000,000
20	Bulgarian	bg	7,800,000