cyroxx/check_pa
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

fix: update tokenization regex to handle hyphens in transcription
Some checks failed
Python tests / tests (push) Failing after 2m5s
Details

Browse Source
This commit is contained in:
cyroxx
2026-02-05 02:35:29 +01:00
parent 0cb53b1822
commit 1943042a42
1 changed files with 1 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
transcription.py
View File

@@ -50,7 +50,7 @@ SPOKEN_TO_CHAR = {
def _normalize_transcription(raw_text): def _normalize_transcription(raw_text):
"""Wandelt eine Whisper-Transkription in die tatsächlichen Captcha-Zeichen um.""" """Wandelt eine Whisper-Transkription in die tatsächlichen Captcha-Zeichen um."""
# Entferne Satzzeichen und splitte in Tokens # Entferne Satzzeichen und splitte in Tokens
tokens = re.split(r'[,.\s]+', raw_text.lower().strip()) tokens = re.split(r'[-,\.\s]+', raw_text.lower().strip())
result = [] result = []
for token in tokens: for token in tokens:
if not token: if not token: