keboss/opencode
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Add FastAPI backend with DZI viewer and feedback system

Browse Source 
- FastAPI app with SQLite DB (projects, pages, issues, feedback)
- OpenSeadragon DZI viewer with inline SVG overlays
- Dashboard: upload, project list, tiling toggle, review mode
- Pipeline integration: tiling OCR → layout → elements → rules QC → DZI → DB
- Feedback collection: true_positive / false_positive / not_sure per issue
This commit is contained in:
Кирилл Блинов 2026-06-01 12:29:41 +03:00
parent feeb02242b
commit f37c477a0a
14 changed files with 2379 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

122
backend/README.md Normal file
View File

@ -0,0 +1,122 @@
# Blueprint QC Backend
FastAPI backend для сбора, хранения и разметки замечаний QC по чертежам.
## Архитектура
```
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
│ React UI │────▶│ FastAPI │────▶│ SQLite │
│ (viewer) │◀────│ Backend │◀────│ Database │
└─────────────┘ └──────────────┘ └─────────────┘
│
▼
┌──────────────┐
│ OCR + QC │
│ Pipeline │
└──────────────┘
```
## Установка
```bash
cd backend
pip install -r requirements.txt
```
## Запуск
```bash
python -m uvicorn app.main:app --reload --port 8000
```
API будет доступен на http://localhost:8000
Документация (Swagger): http://localhost:8000/docs
## API Endpoints
### Проекты
| Method | Endpoint | Описание |
|--------|----------|----------|
| POST | `/api/projects/upload` | Загрузка PDF, запуск обработки |
| GET | `/api/projects` | Список проектов |
| GET | `/api/projects/{id}` | Детали проекта |
### Замечания (Issues)
| Method | Endpoint | Описание |
|--------|----------|----------|
| GET | `/api/projects/{id}/issues` | Замечания проекта |
| GET | `/api/issues/{id}` | Одно замечание |
### Feedback (разметка)
| Method | Endpoint | Описание |
|--------|----------|----------|
| POST | `/api/feedback` | Отметить TP/FP |
| GET | `/api/feedback/stats` | Статистика |
### Обучение
| Method | Endpoint | Описание |
|--------|----------|----------|
| GET | `/api/training/data` | Данные для обучения |
| POST | `/api/training/export` | Скачать JSON |
| GET | `/api/stats` | Общая статистика |
### Viewer
| Method | Endpoint | Описание |
|--------|----------|----------|
| GET | `/viewer/{project_id}/{page}` | HTML viewer |
| GET | `/viewer_tiles/{project_id}/{path}` | DZI тайлы |
## Workflow
1. **Загрузка**: `POST /api/projects/upload` с PDF
2. **Обработка**: Backend запускает OCR → QC → DZI (фоново)
3. **Просмотр**: `GET /viewer/{id}/{page}` — viewer с overlay
4. **Разметка**: Пользователь кликает замечания → `POST /api/feedback` (is_true_positive: true/false)
5. **Обучение**: `GET /api/training/data` — экспорт размеченных данных
## Feedback Schema
```json
{
"issue_id": 123,
"is_true_positive": true, // true = реальная проблема, false = ложное срабатывание
"comment": "Размер действительно плохо читается",
"action_taken": "fixed" // fixed / ignored / not_sure
}
```
## Training Data
Каждый размеченный пример содержит:
- `bbox`: координаты на PNG
- `issue_type`: тип проблемы
- `is_true_positive`: метка от пользователя
- `image_path`: путь к PNG страницы
- `dimension_text`: текст размера (если есть)
- `confidence`: OCR confidence
Накопив 100-200 размеченных примеров, можно:
1. Fine-tune VLM (few-shot prompting)
2. Обучить YOLO-детектор под ваши типы чертежей
3. Дообучить правила QC (эвристики)
## Модели БД
### Project
- id, name, pdf_filename, status, created_at, output_folder
### Page
- id, project_id, page_number, png_path, ocr_data, vlm_description
### Issue
- id, project_id, page_id, issue_type, severity, message, bbox, dimension_text, confidence
### Feedback
- id, issue_id, is_true_positive, comment, action_taken, created_at

1
backend/__init__.py Normal file
View File

@ -0,0 +1 @@
# backend/__init__.py

1
backend/app/__init__.py Normal file
View File

@ -0,0 +1 @@
# app/__init__.py

192
backend/app/crud.py Normal file
View File

@ -0,0 +1,192 @@
# app/crud.py
from sqlalchemy.orm import Session, joinedload
from sqlalchemy import func
from typing import List, Optional
import app.models as models
import app.schemas as schemas
# ---------- Project ----------
def create_project(db: Session, pdf_filename: str, name: Optional[str] = None) -> models.Project:
project = models.Project(
name=name or pdf_filename,
pdf_filename=pdf_filename,
status="uploaded"
)
db.add(project)
db.commit()
db.refresh(project)
return project
def get_project(db: Session, project_id: int) -> Optional[models.Project]:
return db.query(models.Project).options(
joinedload(models.Project.pages),
joinedload(models.Project.issues).joinedload(models.Issue.feedback)
).filter(models.Project.id == project_id).first()
def get_projects(db: Session, skip: int = 0, limit: int = 100) -> List[models.Project]:
return db.query(models.Project).options(
joinedload(models.Project.pages),
joinedload(models.Project.issues).joinedload(models.Issue.feedback)
).order_by(models.Project.created_at.desc()).offset(skip).limit(limit).all()
def update_project_status(db: Session, project_id: int, status: str, error_message: str = None, output_folder: str = None):
project = get_project(db, project_id)
if project:
project.status = status
if error_message is not None:
project.error_message = error_message
# Очищаем ошибку при новом запуске или успехе
if status in ("processing", "completed") and error_message is None:
project.error_message = None
if output_folder:
project.output_folder = output_folder
if status == "completed":
from datetime import datetime
project.completed_at = datetime.utcnow()
db.commit()
db.refresh(project)
return project
# ---------- Page ----------
def create_page(db: Session, project_id: int, page_number: int, **kwargs) -> models.Page:
page = models.Page(project_id=project_id, page_number=page_number, **kwargs)
db.add(page)
db.commit()
db.refresh(page)
return page
def get_page_by_number(db: Session, project_id: int, page_number: int) -> Optional[models.Page]:
return db.query(models.Page).filter(
models.Page.project_id == project_id,
models.Page.page_number == page_number
).first()
# ---------- Issue ----------
def create_issue(db: Session, project_id: int, page_id: Optional[int], **kwargs) -> models.Issue:
issue = models.Issue(project_id=project_id, page_id=page_id, **kwargs)
db.add(issue)
db.commit()
db.refresh(issue)
return issue
def get_issues(db: Session, project_id: Optional[int] = None, page_id: Optional[int] = None,
severity: Optional[str] = None, issue_type: Optional[str] = None,
has_feedback: Optional[bool] = None, skip: int = 0, limit: int = 1000):
query = db.query(models.Issue)
if project_id:
query = query.filter(models.Issue.project_id == project_id)
if page_id:
query = query.filter(models.Issue.page_id == page_id)
if severity:
query = query.filter(models.Issue.severity == severity)
if issue_type:
query = query.filter(models.Issue.issue_type == issue_type)
if has_feedback is not None:
if has_feedback:
query = query.filter(models.Issue.feedback != None)
else:
query = query.filter(models.Issue.feedback == None)
return query.order_by(models.Issue.created_at.desc()).offset(skip).limit(limit).all()
def get_issue(db: Session, issue_id: int) -> Optional[models.Issue]:
return db.query(models.Issue).filter(models.Issue.id == issue_id).first()
# ---------- Feedback ----------
def create_feedback(db: Session, feedback: schemas.FeedbackCreate) -> models.Feedback:
# Удалить старый feedback если есть
existing = db.query(models.Feedback).filter(models.Feedback.issue_id == feedback.issue_id).first()
if existing:
db.delete(existing)
db.commit()
db_feedback = models.Feedback(**feedback.dict())
db.add(db_feedback)
db.commit()
db.refresh(db_feedback)
return db_feedback
def get_feedback_stats(db: Session, project_id: Optional[int] = None):
query = db.query(models.Feedback)
if project_id:
query = query.join(models.Issue).filter(models.Issue.project_id == project_id)
total = query.count()
true_positive = query.filter(models.Feedback.is_true_positive == True).count()
false_positive = query.filter(models.Feedback.is_true_positive == False).count()
unreviewed = query.filter(models.Feedback.is_true_positive == None).count()
accuracy = true_positive / (true_positive + false_positive) if (true_positive + false_positive) > 0 else None
return {
"total": total,
"true_positive": true_positive,
"false_positive": false_positive,
"unreviewed": unreviewed,
"accuracy_estimate": round(accuracy, 3) if accuracy else None
}
# ---------- Stats ----------
def get_stats(db: Session) -> schemas.StatsResponse:
total_projects = db.query(models.Project).count()
total_issues = db.query(models.Issue).count()
# По типам
issue_types = db.query(models.Issue.issue_type, func.count(models.Issue.id)).group_by(models.Issue.issue_type).all()
issues_by_type = {t[0]: t[1] for t in issue_types}
# Feedback
fb_stats = get_feedback_stats(db)
return schemas.StatsResponse(
total_projects=total_projects,
total_issues=total_issues,
issues_by_type=issues_by_type,
feedback_stats={
"true_positive": fb_stats["true_positive"],
"false_positive": fb_stats["false_positive"],
"unreviewed": fb_stats["unreviewed"]
},
accuracy_estimate=fb_stats["accuracy_estimate"]
)
# ---------- Training Data ----------
def export_training_data(db: Session, project_id: Optional[int] = None,
only_labeled: bool = True) -> List[schemas.TrainingSample]:
"""Экспорт данных для обучения ML-модели."""
query = db.query(models.Issue)
if project_id:
query = query.filter(models.Issue.project_id == project_id)
if only_labeled:
query = query.join(models.Feedback)
issues = query.all()
samples = []
for issue in issues:
sample = schemas.TrainingSample(
issue_id=issue.id,
issue_type=issue.issue_type,
severity=issue.severity,
message=issue.message,
bbox={
"x1": issue.bbox_x1,
"y1": issue.bbox_y1,
"x2": issue.bbox_x2,
"y2": issue.bbox_y2
},
dimension_text=issue.dimension_text,
confidence=issue.confidence,
page_number=issue.page.page_number if issue.page else None,
is_true_positive=issue.feedback.is_true_positive if issue.feedback else None,
image_path=issue.page.png_path if issue.page else None,
label=f"{'good' if issue.feedback and issue.feedback.is_true_positive else 'bad'}_{issue.issue_type.lower()}" if issue.feedback else None
)
samples.append(sample)
return samples

23
backend/app/database.py Normal file
View File

@ -0,0 +1,23 @@
# app/database.py
import os
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import sessionmaker
SQLALCHEMY_DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:///./blueprint_qc.db")
engine = create_engine(
SQLALCHEMY_DATABASE_URL,
connect_args={"check_same_thread": False} if SQLALCHEMY_DATABASE_URL.startswith("sqlite") else {}
)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
Base = declarative_base()
def get_db():
db = SessionLocal()
try:
yield db
finally:
db.close()

401
backend/app/main.py Normal file
View File

@ -0,0 +1,401 @@
# app/main.py
"""
FastAPI backend для Blueprint QC Service.
API Endpoints:
- POST /api/projects/upload — загрузка PDF
- GET /api/projects — список проектов
- GET /api/projects/{id} — детали проекта
- GET /api/projects/{id}/issues — замечания проекта
- POST /api/feedback — отметить замечание (TP/FP)
- GET /api/feedback/stats — статистика feedback
- GET /api/training/data — экспорт данных для обучения
- GET /api/stats — общая статистика
- GET /viewer/{project_id}/{page} — HTML viewer (серверный, без file://)
"""
import os
import sys
import re
import json
import shutil
from pathlib import Path
from typing import List, Optional
from datetime import datetime
from fastapi import FastAPI, File, UploadFile, Depends, HTTPException, Query
from fastapi.responses import HTMLResponse, FileResponse, JSONResponse
from fastapi.middleware.cors import CORSMiddleware
from fastapi.staticfiles import StaticFiles
from sqlalchemy.orm import Session
# Добавить пути
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
sys.path.insert(0, str(Path(__file__).parent))
from app.database import engine, get_db, Base
from app import models, schemas, crud, processing
# Создать таблицы
Base.metadata.create_all(bind=engine)
# Создать папку static если нет
STATIC_DIR = Path(__file__).parent.parent / "static"
STATIC_DIR.mkdir(exist_ok=True)
app = FastAPI(
title="Blueprint QC API",
description="Сервис автоматической проверки чертежей и сбора данных для обучения",
version="0.1.0"
)
# CORS
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
# Пути
# Абсолютные пути (backend запускается из разных директорий)
BASE_DIR = Path(__file__).parent.parent # backend/
UPLOAD_DIR = BASE_DIR / "uploads"
OUTPUT_DIR = BASE_DIR / "outputs"
UPLOAD_DIR.mkdir(exist_ok=True)
OUTPUT_DIR.mkdir(exist_ok=True)
# ==================== PROJECTS ====================
@app.post("/api/projects/upload", response_model=schemas.ProjectResponse)
async def upload_pdf(
file: UploadFile = File(...),
name: Optional[str] = None,
db: Session = Depends(get_db)
):
"""Загрузка PDF файла. Запускает фоновую обработку."""
if not file.filename.endswith('.pdf'):
raise HTTPException(400, "Only PDF files allowed")
# Сохранить файл
safe_name = Path(file.filename).name
pdf_path = UPLOAD_DIR / f"{datetime.now().strftime('%Y%m%d_%H%M%S')}_{safe_name}"
with open(pdf_path, "wb") as f:
shutil.copyfileobj(file.file, f)
# Создать проект в БД (статус = uploaded, анализ запускается вручную)
project = crud.create_project(db, pdf_filename=safe_name, name=name or safe_name)
# НЕ запускаем обработку автоматически — пользователь жмет "Анализировать"
# вручную, когда готов
return project
@app.get("/api/projects", response_model=List[schemas.ProjectDetail])
def list_projects(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
"""Список всех проектов со страницами и замечаниями."""
return crud.get_projects(db, skip=skip, limit=limit)
@app.get("/api/projects/{project_id}", response_model=schemas.ProjectDetail)
def get_project(project_id: int, db: Session = Depends(get_db)):
"""Детали проекта со страницами и замечаниями."""
project = crud.get_project(db, project_id)
if not project:
raise HTTPException(404, "Project not found")
return project
@app.delete("/api/projects/{project_id}")
def delete_project(project_id: int, db: Session = Depends(get_db)):
"""Удалить проект и все связанные данные."""
project = crud.get_project(db, project_id)
if not project:
raise HTTPException(404, "Project not found")
# Удалить файлы проекта
if project.output_folder:
import shutil
try:
shutil.rmtree(project.output_folder, ignore_errors=True)
except:
pass
db.delete(project)
db.commit()
return {"status": "deleted", "project_id": project_id}
@app.post("/api/projects/{project_id}/analyze")
def analyze_project(project_id: int, use_tiling: bool = False, db: Session = Depends(get_db)):
"""Запустить анализ (OCR + Layout + QC + DZI) для проекта."""
project = crud.get_project(db, project_id)
if not project:
raise HTTPException(404, "Project not found")
if project.status == "processing":
return {"status": "already_processing", "project_id": project_id}
if project.status == "completed":
pass
pdf_files = list(UPLOAD_DIR.glob(f"*_{project.pdf_filename}"))
if not pdf_files:
pdf_files = list(UPLOAD_DIR.glob(project.pdf_filename))
if not pdf_files:
raise HTTPException(404, f"PDF file not found in {UPLOAD_DIR}")
pdf_path = pdf_files[0]
crud.update_project_status(db, project_id, "processing")
import threading
def run_in_background():
db_local = next(get_db())
try:
processing.run_pipeline(project.id, pdf_path, OUTPUT_DIR, db_local, use_tiling=use_tiling)
except Exception as e:
print(f"[ERROR] Pipeline failed for project {project.id}: {e}")
import traceback
traceback.print_exc()
crud.update_project_status(db_local, project.id, "error", error_message=str(e))
finally:
db_local.close()
thread = threading.Thread(target=run_in_background)
thread.start()
return {"status": "processing_started", "project_id": project_id, "ocr_engine": "tiling" if use_tiling else "standard"}
# ==================== ISSUES ====================
@app.get("/api/projects/{project_id}/issues")
def get_issues(
project_id: int,
severity: Optional[str] = Query(None, description="error / warning / info"),
issue_type: Optional[str] = Query(None),
has_feedback: Optional[bool] = Query(None, description="Filter by feedback presence"),
db: Session = Depends(get_db)
):
"""Получить замечания проекта."""
issues = crud.get_issues(
db, project_id=project_id, severity=severity,
issue_type=issue_type, has_feedback=has_feedback
)
# Build response with page_number manually
result = []
for issue in issues:
item = {
"id": issue.id,
"project_id": issue.project_id,
"page_id": issue.page_id,
"page_number": issue.page.page_number if issue.page else None,
"issue_type": issue.issue_type,
"severity": issue.severity,
"message": issue.message,
"bbox_x1": issue.bbox_x1,
"bbox_y1": issue.bbox_y1,
"bbox_x2": issue.bbox_x2,
"bbox_y2": issue.bbox_y2,
"dimension_text": issue.dimension_text,
"confidence": issue.confidence,
"created_at": issue.created_at.isoformat() if issue.created_at else None,
"feedback": {
"id": issue.feedback.id,
"issue_id": issue.feedback.issue_id,
"is_true_positive": issue.feedback.is_true_positive,
"comment": issue.feedback.comment,
"action_taken": issue.feedback.action_taken,
"created_at": issue.feedback.created_at.isoformat() if issue.feedback.created_at else None
} if issue.feedback else None
}
result.append(item)
return result
@app.get("/api/issues/{issue_id}", response_model=schemas.IssueResponse)
def get_issue(issue_id: int, db: Session = Depends(get_db)):
"""Одно замечание."""
issue = crud.get_issue(db, issue_id)
if not issue:
raise HTTPException(404, "Issue not found")
return issue
# ==================== FEEDBACK ====================
@app.post("/api/feedback", response_model=schemas.FeedbackResponse)
def submit_feedback(feedback: schemas.FeedbackCreate, db: Session = Depends(get_db)):
"""
Отправить feedback по замечанию.
is_true_positive:
- true = реальная проблема (правильное срабатывание)
- false = ложное срабатывание (false positive)
- null = не уверен
action_taken: fixed / ignored / not_sure
"""
issue = crud.get_issue(db, feedback.issue_id)
if not issue:
raise HTTPException(404, "Issue not found")
return crud.create_feedback(db, feedback)
@app.get("/api/feedback/stats")
def feedback_stats(project_id: Optional[int] = None, db: Session = Depends(get_db)):
"""Статистика feedback."""
return crud.get_feedback_stats(db, project_id=project_id)
# ==================== VIEWER ====================
@app.get("/viewer/{project_id}/{page_number}", response_class=HTMLResponse)
def get_viewer(project_id: int, page_number: int, db: Session = Depends(get_db)):
"""HTML viewer с overlay замечаний + feedback buttons."""
project = crud.get_project(db, project_id)
if not project or not project.output_folder:
raise HTTPException(404, "Project or output not found")
# ВСЕГДА перегенерировать viewer для нужной страницы
# (иначе тайлы указывают на другую страницу)
result = processing.generate_viewer_html(db, project_id, page_number)
if not result:
raise HTTPException(404, "Viewer not available")
viewer_path = Path(result)
content = viewer_path.read_text(encoding="utf-8")
# Подменить пути тайлов — теперь они будут правильные
# (generate_web_viewer.py уже сгенерировал для page_{page_number:03d}_files/)
content = content.replace(
'Url: "./page_',
f'Url: "/viewer_tiles/{project_id}/page_'
)
# Внедрить project_id и page info для навигации
total_pages = len(project.pages) if project.pages else page_number
content = content.replace('const PROJECT_ID = null;', f'const PROJECT_ID = {project_id};')
content = content.replace('const TOTAL_PAGES = null;', f'const TOTAL_PAGES = {total_pages};')
content = content.replace('const CURRENT_PAGE = null;', f'const CURRENT_PAGE = {page_number};')
# Обновить счётчик страниц в навбаре
content = content.replace(
f'<span id="totalPages">?</span>',
f'<span id="totalPages">{total_pages}</span>'
)
# Получить issue IDs из БД для этой страницы
page = crud.get_page_by_number(db, project_id, page_number)
if page:
issues = crud.get_issues(db, project_id=project_id, page_id=page.id)
issue_db_ids = [str(i.id) for i in issues]
# Внедрить DB IDs в HTML
content = content.replace('data-has-api="false"', 'data-has-api="true"')
def inject_db_id(match):
idx = int(match.group(1)) - 1
if idx < len(issue_db_ids):
return match.group(0) + f' data-db-id="{issue_db_ids[idx]}"'
return match.group(0)
content = re.sub(r'data-id="(\d+)"', inject_db_id, content)
return HTMLResponse(content=content)
@app.get("/viewer_tiles/{project_id}/{filename:path}")
def get_tile(project_id: int, filename: str, db: Session = Depends(get_db)):
"""Отдаёт DZI тайлы."""
project = crud.get_project(db, project_id)
if not project or not project.output_folder:
raise HTTPException(404)
tile_path = Path(project.output_folder) / filename
if tile_path.exists() and str(tile_path).startswith(str(project.output_folder)):
return FileResponse(tile_path)
raise HTTPException(404)
# ==================== TRAINING DATA ====================
@app.get("/api/training/data", response_model=schemas.TrainingDataExport)
def export_training_data(
project_id: Optional[int] = Query(None),
only_labeled: bool = Query(True),
format: str = Query("json", description="json / yolo / csv"),
db: Session = Depends(get_db)
):
"""
Экспорт данных для обучения ML-модели.
only_labeled=true — только размеченные feedback'ом замечания.
"""
samples = crud.export_training_data(db, project_id=project_id, only_labeled=only_labeled)
if format == "json":
return schemas.TrainingDataExport(
total_samples=len(samples),
labeled_samples=len([s for s in samples if s.is_true_positive is not None]),
samples=samples,
export_format="json"
)
# TODO: YOLO / CSV форматы
raise HTTPException(400, f"Format {format} not yet implemented")
@app.post("/api/training/export")
def download_training_export(
project_id: Optional[int] = None,
only_labeled: bool = True,
db: Session = Depends(get_db)
):
"""Скачать training data как JSON файл."""
samples = crud.export_training_data(db, project_id=project_id, only_labeled=only_labeled)
export = {
"generated_at": datetime.utcnow().isoformat(),
"total_samples": len(samples),
"samples": [s.dict() for s in samples]
}
return JSONResponse(content=export, media_type="application/json")
# ==================== STATS ====================
@app.get("/api/stats", response_model=schemas.StatsResponse)
def get_stats(db: Session = Depends(get_db)):
"""Общая статистика системы."""
return crud.get_stats(db)
# ==================== HEALTH ====================
@app.get("/api/health")
def health_check():
return {"status": "ok", "version": "0.1.0"}
# ==================== STATIC (для демо) ====================
# Подключить статику
try:
app.mount("/", StaticFiles(directory=str(STATIC_DIR), html=True), name="static")
except RuntimeError:
pass # Already mounted
if __name__ == "__main__":
import uvicorn
uvicorn.run(app, host="0.0.0.0", port=8000)

95
backend/app/models.py Normal file
View File

@ -0,0 +1,95 @@
# app/models.py
from sqlalchemy import Column, Integer, String, Float, DateTime, Text, ForeignKey, JSON, Boolean
from sqlalchemy.orm import relationship
from datetime import datetime
from app.database import Base
class Project(Base):
"""Проект — один загруженный PDF."""
__tablename__ = "projects"
id = Column(Integer, primary_key=True, index=True)
name = Column(String, index=True)
pdf_filename = Column(String, nullable=False)
status = Column(String, default="uploaded") # uploaded / processing / completed / error
created_at = Column(DateTime, default=datetime.utcnow)
completed_at = Column(DateTime, nullable=True)
error_message = Column(Text, nullable=True)
# Пути к файлам
output_folder = Column(String, nullable=True)
pages = relationship("Page", back_populates="project", cascade="all, delete-orphan")
issues = relationship("Issue", back_populates="project", cascade="all, delete-orphan")
class Page(Base):
"""Страница PDF."""
__tablename__ = "pages"
id = Column(Integer, primary_key=True, index=True)
project_id = Column(Integer, ForeignKey("projects.id"))
page_number = Column(Integer, nullable=False)
png_path = Column(String, nullable=True)
dzi_path = Column(String, nullable=True)
ocr_data = Column(JSON, nullable=True) # full_ocr_results для этой страницы
vlm_description = Column(Text, nullable=True)
width = Column(Integer, nullable=True)
height = Column(Integer, nullable=True)
project = relationship("Project", back_populates="pages")
issues = relationship("Issue", back_populates="page")
class Issue(Base):
"""Замечание QC — одна проблема на чертеже."""
__tablename__ = "issues"
id = Column(Integer, primary_key=True, index=True)
project_id = Column(Integer, ForeignKey("projects.id"))
page_id = Column(Integer, ForeignKey("pages.id"))
issue_type = Column(String, nullable=False) # DIMENSION_OVERLAP, LOW_CONFIDENCE, etc.
severity = Column(String, nullable=False) # error / warning / info
message = Column(Text, nullable=False)
# Координаты bbox на PNG (в пикселях)
bbox_x1 = Column(Float, nullable=True)
bbox_y1 = Column(Float, nullable=True)
bbox_x2 = Column(Float, nullable=True)
bbox_y2 = Column(Float, nullable=True)
# Дополнительные данные
dimension_text = Column(String, nullable=True) # Текст размера (если применимо)
confidence = Column(Float, nullable=True) # OCR confidence
extra_data = Column(JSON, nullable=True) # Всё остальное
source = Column(String, nullable=True) # "rules" или "vlm"
created_at = Column(DateTime, default=datetime.utcnow)
project = relationship("Project", back_populates="issues")
page = relationship("Page", back_populates="issues")
feedback = relationship("Feedback", back_populates="issue", uselist=False)
class Feedback(Base):
"""Feedback проектировщика — правда ли это замечание."""
__tablename__ = "feedback"
id = Column(Integer, primary_key=True, index=True)
issue_id = Column(Integer, ForeignKey("issues.id"), unique=True)
# True = реальная проблема, False = ложное срабатывание, None = не размечено
is_true_positive = Column(Boolean, nullable=True)
# Почему (опционально)
comment = Column(Text, nullable=True)
# Что сделал проектировщик
action_taken = Column(String, nullable=True) # fixed / ignored / not_sure
created_at = Column(DateTime, default=datetime.utcnow)
user_id = Column(String, nullable=True) # Для многопользовательского режима
issue = relationship("Issue", back_populates="feedback")

217
backend/app/processing.py Normal file
View File

@ -0,0 +1,217 @@
# app/processing.py
"""
Интеграция с существующими скриптами OCR, QC, DZI.
Запускает pipeline в фоне и сохраняет результаты в БД.
"""
import os
import sys
import json
import re
import subprocess
import shutil
from pathlib import Path
from typing import Optional
from sqlalchemy.orm import Session
# Добавить корень проекта в path для импорта скриптов
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
import app.crud as crud
import app.models as models
def run_pipeline(project_id: int, pdf_path: Path, output_base: Path, db: Session, use_tiling: bool = False):
"""Запускает полный pipeline: PDF → OCR → Layout → Elements → QC → DZI → БД."""
project = crud.get_project(db, project_id)
if not project:
return
crud.update_project_status(db, project_id, "processing")
try:
# 1. Создать output папку
output_folder = output_base / f"project_{project_id}"
output_folder.mkdir(parents=True, exist_ok=True)
script_dir = Path(__file__).parent.parent.parent # backend/app -> backend -> opencode
# 2. OCR + PNG (RapidOCR или Tiling OCR)
cmd = [sys.executable, str(script_dir / "process_any_pdf.py"), str(pdf_path), str(output_folder)]
if use_tiling:
cmd.append("--use-tiling")
print("[INFO] Tiling OCR enabled")
_run_command(cmd, cwd=str(script_dir))
# 3. Layout Detection для каждой оригинальной страницы
# Только page_NNN.png, исключая визуализации (_dims, _layout и т.д.)
page_pngs = sorted([p for p in output_folder.glob("page_*.png")
if not any(suffix in p.stem for suffix in ["_dims", "_layout", "_detected", "_preproc", "_ocr_compare"])])
for png in page_pngs:
ocr_json = output_folder / "full_ocr_results.json"
if ocr_json.exists():
try:
_run_command([
sys.executable, str(script_dir / "layout_detector.py"),
str(png), str(ocr_json)
], cwd=str(script_dir))
print(f"[INFO] Layout detection done for {png.name}")
except Exception as e:
print(f"[WARN] Layout detection failed for {png.name}: {e}")
# 4. Multi-Element Extraction (dimensions, positions, GOSTs, etc.)
for png in page_pngs:
ocr_json = output_folder / "full_ocr_results.json"
layout_json = output_folder / "layout.json"
if ocr_json.exists() and layout_json.exists():
try:
_run_command([
sys.executable, str(script_dir / "multi_element_extractor.py"),
str(png), str(ocr_json), str(layout_json)
], cwd=str(script_dir))
print(f"[INFO] Element extraction done for {png.name}")
except Exception as e:
print(f"[WARN] Element extraction failed for {png.name}: {e}")
# 5. QC (dimension_qc_checker.py) — правила
_run_command([
sys.executable, str(script_dir / "dimension_qc_checker.py"),
str(output_folder)
], cwd=str(script_dir))
# 6. DZI для каждой страницы
for png in page_pngs:
_run_command([
sys.executable, str(script_dir / "generate_dzi.py"),
str(png)
], cwd=str(script_dir))
# 7. Проверить результаты
ocr_path = output_folder / "full_ocr_results.json"
if not ocr_path.exists():
raise RuntimeError(f"OCR results not generated: {ocr_path}")
# 8. Загрузить в БД
_import_results(db, project_id, output_folder)
crud.update_project_status(db, project_id, "completed", output_folder=str(output_folder))
except Exception as e:
crud.update_project_status(db, project_id, "error", error_message=str(e))
raise
def _run_command(cmd: list, cwd: Optional[Path] = None):
"""Запускает команду, проверяет exit code, выбрасывает исключение при ошибке."""
result = subprocess.run(
cmd,
capture_output=True,
text=True,
cwd=cwd
)
if result.returncode != 0:
stderr = result.stderr[:1000]
stdout = result.stdout[:500]
raise RuntimeError(f"Command failed ({result.returncode}): {' '.join(cmd)}\nSTDERR: {stderr}\nSTDOUT: {stdout}")
return result
def _import_results(db: Session, project_id: int, output_folder: Path):
"""Импорт OCR и QC результатов в БД. Очищает старые данные проекта перед импортом."""
# Очистить старые страницы и замечания (cascade удалит issues и feedback)
db.query(models.Page).filter(models.Page.project_id == project_id).delete(synchronize_session=False)
db.query(models.Issue).filter(models.Issue.project_id == project_id).delete(synchronize_session=False)
db.commit()
# Загрузить OCR
ocr_path = output_folder / "full_ocr_results.json"
if ocr_path.exists():
ocr = json.loads(ocr_path.read_text(encoding="utf-8"))
for page_data in ocr.get("pages", []):
page_num = page_data["page_number"]
png_path = output_folder / f"page_{page_num:03d}.png"
page = crud.create_page(
db, project_id=project_id, page_number=page_num,
png_path=str(png_path) if png_path.exists() else None,
ocr_data=page_data
)
# Загрузить VLM extraction descriptions в pages
vlm_path = output_folder / "vlm_extraction.json"
vlm_data = {}
if vlm_path.exists():
vlm_data = json.loads(vlm_path.read_text(encoding="utf-8"))
# Обновить vlm_description для каждой страницы
for img_name, extraction in vlm_data.items():
page_num_match = re.search(r"page_(\d+)", img_name)
if page_num_match:
page_num = int(page_num_match.group(1))
page = crud.get_page_by_number(db, project_id, page_num)
if page:
desc = extraction.get("description", "")
page.vlm_description = desc[:2000] if desc else None # ограничим длину
db.commit()
# Загрузить QC issues (только rules — VLM issues удалены как ненадёжные)
qc_path = output_folder / "dimension_qc_report.json"
if qc_path.exists():
qc = json.loads(qc_path.read_text(encoding="utf-8"))
for severity in ["errors", "warnings", "infos"]:
for item in qc.get(severity, []):
page_num = item.get("page")
if not page_num:
continue
page = crud.get_page_by_number(db, project_id, page_num)
# Извлечь bbox
bbox = item.get("bbox") or item.get("bbox1") or item.get("bbox_dim")
x1 = y1 = x2 = y2 = None
if bbox:
if isinstance(bbox[0], list):
xs = [p[0] for p in bbox]
ys = [p[1] for p in bbox]
x1, y1, x2, y2 = min(xs), min(ys), max(xs), max(ys)
else:
x1, y1, x2, y2 = bbox[0], bbox[1], bbox[2], bbox[3]
crud.create_issue(
db, project_id=project_id, page_id=page.id if page else None,
issue_type=item.get("type", "UNKNOWN"),
severity=item.get("severity", "warning"),
message=item.get("message", ""),
bbox_x1=x1, bbox_y1=y1, bbox_x2=x2, bbox_y2=y2,
dimension_text=item.get("text"),
confidence=item.get("confidence"),
source="rules",
extra_data={k: v for k, v in item.items() if k not in ["type", "severity", "message", "page", "text", "confidence", "bbox", "bbox1", "bbox2", "bbox_dim", "source"]}
)
def generate_viewer_html(db: Session, project_id: int, page_number: int) -> Optional[str]:
"""Генерирует HTML viewer для конкретной страницы."""
project = crud.get_project(db, project_id)
if not project or not project.output_folder:
return None
output_folder = Path(project.output_folder)
# Перегенерировать viewer для нужной страницы
# Запускаем из папки, где находится generate_web_viewer.py
script_dir = Path(__file__).parent.parent.parent # backend/app -> backend -> opencode
result = _run_command([
sys.executable, str(script_dir / "generate_web_viewer.py"), str(output_folder), str(page_number)
], cwd=str(script_dir))
if result.returncode != 0:
print(f"[ERROR] generate_web_viewer.py failed: {result.stderr[:500]}")
return None
viewer_path = output_folder / "web_viewer" / "index.html"
if viewer_path.exists():
return str(viewer_path)
return None

137
backend/app/schemas.py Normal file
View File

@ -0,0 +1,137 @@
# app/schemas.py
from pydantic import BaseModel
from typing import List, Optional, Dict, Any
from datetime import datetime
# ---------- Project ----------
class ProjectBase(BaseModel):
name: Optional[str] = None
class ProjectCreate(ProjectBase):
pdf_filename: str
class ProjectResponse(ProjectBase):
id: int
pdf_filename: str
status: str
created_at: datetime
completed_at: Optional[datetime] = None
error_message: Optional[str] = None
class Config:
from_attributes = True
class ProjectDetail(ProjectResponse):
pages: List["PageResponse"] = []
issues: List["IssueResponse"] = []
class Config:
from_attributes = True
# ---------- Page ----------
class PageBase(BaseModel):
page_number: int
width: Optional[int] = None
height: Optional[int] = None
class PageResponse(PageBase):
id: int
png_path: Optional[str] = None
vlm_description: Optional[str] = None
ocr_data: Optional[Dict[str, Any]] = None
issue_count: int = 0
class Config:
from_attributes = True
# ---------- Issue ----------
class IssueBase(BaseModel):
issue_type: str
severity: str
message: str
page_number: Optional[int] = None
class IssueCreate(IssueBase):
bbox_x1: Optional[float] = None
bbox_y1: Optional[float] = None
bbox_x2: Optional[float] = None
bbox_y2: Optional[float] = None
dimension_text: Optional[str] = None
confidence: Optional[float] = None
extra_data: Optional[Dict[str, Any]] = None
class IssueResponse(IssueBase):
id: int
project_id: int
bbox_x1: Optional[float] = None
bbox_y1: Optional[float] = None
bbox_x2: Optional[float] = None
bbox_y2: Optional[float] = None
dimension_text: Optional[str] = None
confidence: Optional[float] = None
source: Optional[str] = None
created_at: datetime
feedback: Optional["FeedbackResponse"] = None
page_id: Optional[int] = None
class Config:
from_attributes = True
# ---------- Feedback ----------
class FeedbackCreate(BaseModel):
issue_id: int
is_true_positive: Optional[bool] = None
comment: Optional[str] = None
action_taken: Optional[str] = None # fixed / ignored / not_sure
class FeedbackResponse(BaseModel):
id: int
issue_id: int
is_true_positive: Optional[bool] = None
comment: Optional[str] = None
action_taken: Optional[str] = None
created_at: datetime
class Config:
from_attributes = True
# ---------- Stats ----------
class StatsResponse(BaseModel):
total_projects: int
total_issues: int
issues_by_type: Dict[str, int]
feedback_stats: Dict[str, int] # true_positive, false_positive, unreviewed
accuracy_estimate: Optional[float] = None # Доля правильных срабатываний
# ---------- Training Data ----------
class TrainingSample(BaseModel):
"""Один пример для обучения ML-модели."""
issue_id: int
issue_type: str
severity: str
message: str
bbox: Dict[str, float] # x1, y1, x2, y2
dimension_text: Optional[str] = None
confidence: Optional[float] = None
page_number: int
is_true_positive: Optional[bool] = None
# Для обучения детектора (YOLO)
image_path: Optional[str] = None # Путь к PNG страницы
crop_path: Optional[str] = None # Вырезанный фрагмент bbox
label: Optional[str] = None # "good_dimension" / "bad_placement" / etc.
class TrainingDataExport(BaseModel):
total_samples: int
labeled_samples: int # Есть is_true_positive
samples: List[TrainingSample]
export_format: str # json / yolo / csv
class Config:
from_attributes = True

223
backend/backend.log Normal file
View File

@ -0,0 +1,223 @@
INFO: Started server process [49915]
INFO: Waiting for application startup.
INFO: Application startup complete.
INFO: Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)
INFO: 127.0.0.1:59330 - "GET /api/health HTTP/1.1" 200 OK
[WARN] Command failed: /opt/homebrew/opt/python@3.11/bin/python3.11 generate_web_viewer.py /Users/kirillblinov/development/opencode/OCR/opencode/output_123 5
stderr: /opt/homebrew/Cellar/python@3.11/3.11.15_1/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python: can't open file '/Users/kirillblinov/development/opencode/OCR/generate_web_viewer.py': [Errno 2] No such file or directory
INFO: 127.0.0.1:59332 - "GET /viewer/1/5 HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/protocols/http/httptools_impl.py", line 421, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/middleware/proxy_headers.py", line 56, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/applications.py", line 1159, in __call__
await super().__call__(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/applications.py", line 90, in __call__
await self.middleware_stack(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/cors.py", line 88, in __call__
await self.app(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 660, in __call__
await self.middleware_stack(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 680, in app
await route.handle(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 134, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 120, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 674, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 330, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/concurrency.py", line 32, in run_in_threadpool
return await anyio.to_thread.run_sync(func)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/to_thread.py", line 63, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 2518, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 1002, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/development/opencode/OCR/opencode/backend/app/main.py", line 285, in get_viewer
content = re.sub(r'data-id="(\d+)"', inject_db_id, content)
^^
NameError: name 're' is not defined
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/14/33_5.png HTTP/1.1" 200 OK
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/14/33_4.png HTTP/1.1" 200 OK
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_2.png HTTP/1.1" 200 OK
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_1.png HTTP/1.1" 200 OK
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_3.png HTTP/1.1" 200 OK
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_0.png HTTP/1.1" 200 OK
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_4.png HTTP/1.1" 200 OK
[WARN] Command failed: /opt/homebrew/opt/python@3.11/bin/python3.11 generate_web_viewer.py /Users/kirillblinov/development/opencode/OCR/opencode/output_123 5
stderr: /opt/homebrew/Cellar/python@3.11/3.11.15_1/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python: can't open file '/Users/kirillblinov/development/opencode/OCR/generate_web_viewer.py': [Errno 2] No such file or directory
INFO: 127.0.0.1:59372 - "GET /viewer/1/5 HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/protocols/http/httptools_impl.py", line 421, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/middleware/proxy_headers.py", line 56, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/applications.py", line 1159, in __call__
await super().__call__(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/applications.py", line 90, in __call__
await self.middleware_stack(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/cors.py", line 88, in __call__
await self.app(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 660, in __call__
await self.middleware_stack(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 680, in app
await route.handle(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 134, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 120, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 674, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 330, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/concurrency.py", line 32, in run_in_threadpool
return await anyio.to_thread.run_sync(func)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/to_thread.py", line 63, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 2518, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 1002, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/development/opencode/OCR/opencode/backend/app/main.py", line 285, in get_viewer
content = re.sub(r'data-id="(\d+)"', inject_db_id, content)
^^
NameError: name 're' is not defined
INFO: 127.0.0.1:59355 - "GET /api/projects HTTP/1.1" 200 OK
INFO: 127.0.0.1:59406 - "GET /api/stats HTTP/1.1" 200 OK
INFO: 127.0.0.1:59355 - "GET /api/projects HTTP/1.1" 200 OK
INFO: 127.0.0.1:59408 - "GET /api/stats HTTP/1.1" 200 OK
[WARN] Command failed: /opt/homebrew/opt/python@3.11/bin/python3.11 generate_web_viewer.py /Users/kirillblinov/development/opencode/OCR/opencode/output_123 5
stderr: /opt/homebrew/Cellar/python@3.11/3.11.15_1/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python: can't open file '/Users/kirillblinov/development/opencode/OCR/generate_web_viewer.py': [Errno 2] No such file or directory
INFO: 127.0.0.1:59425 - "GET /viewer/1/5 HTTP/1.1" 500 Internal Server Error
ERROR: Exception in ASGI application
Traceback (most recent call last):
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/protocols/http/httptools_impl.py", line 421, in run_asgi
result = await app( # type: ignore[func-returns-value]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/middleware/proxy_headers.py", line 56, in __call__
return await self.app(scope, receive, send)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/applications.py", line 1159, in __call__
await super().__call__(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/applications.py", line 90, in __call__
await self.middleware_stack(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 186, in __call__
raise exc
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 164, in __call__
await self.app(scope, receive, _send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/cors.py", line 88, in __call__
await self.app(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
await self.app(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 660, in __call__
await self.middleware_stack(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 680, in app
await route.handle(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 276, in handle
await self.app(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 134, in app
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
raise exc
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
await app(scope, receive, sender)
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 120, in app
response = await f(request)
^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 674, in app
raw_response = await run_endpoint_function(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 330, in run_endpoint_function
return await run_in_threadpool(dependant.call, **values)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/concurrency.py", line 32, in run_in_threadpool
return await anyio.to_thread.run_sync(func)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/to_thread.py", line 63, in run_sync
return await get_async_backend().run_sync_in_worker_thread(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 2518, in run_sync_in_worker_thread
return await future
^^^^^^^^^^^^
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 1002, in run
result = context.run(func, *args)
^^^^^^^^^^^^^^^^^^^^^^^^
File "/Users/kirillblinov/development/opencode/OCR/opencode/backend/app/main.py", line 285, in get_viewer
content = re.sub(r'data-id="(\d+)"', inject_db_id, content)
^^
NameError: name 're' is not defined
INFO: 127.0.0.1:59553 - "GET /viewer_tiles/1/page_005_files/14/18_1.png HTTP/1.1" 200 OK
INFO: 127.0.0.1:59553 - "GET /viewer_tiles/1/page_005_files/14/18_0.png HTTP/1.1" 200 OK
INFO: Shutting down
INFO: Waiting for application shutdown.
INFO: Application shutdown complete.
INFO: Finished server process [49915]

137
backend/import_existing.py Normal file
View File

@ -0,0 +1,137 @@
#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Импорт существующего output_123 в backend БД.
Использование:
python import_existing.py
"""
import sys
import json
from pathlib import Path
# Добавить backend в path
sys.path.insert(0, str(Path(__file__).parent.parent / "backend"))
from app.database import SessionLocal, Base, engine
from app import models, crud, schemas
def import_output_123():
"""Импорт существующего проекта output_123 в БД."""
output_folder = Path("/Users/kirillblinov/development/opencode/OCR/opencode/output_123")
if not output_folder.exists():
print(f"[ERR] Не найден: {output_folder}")
return
db = SessionLocal()
try:
# Создать проект
project = crud.create_project(
db,
pdf_filename="123.pdf",
name="Test Project 123"
)
print(f"[INFO] Создан проект: ID={project.id}")
# Обновить статус
crud.update_project_status(db, project.id, "completed", output_folder=str(output_folder))
# Загрузить OCR
ocr_path = output_folder / "full_ocr_results.json"
if ocr_path.exists():
ocr = json.loads(ocr_path.read_text(encoding="utf-8"))
# Создать страницы
for page_data in ocr.get("pages", []):
page_num = page_data["page_number"]
png_path = output_folder / f"page_{page_num:03d}.png"
from PIL import Image
width = height = None
if png_path.exists():
with Image.open(png_path) as img:
width, height = img.size
page = crud.create_page(
db,
project_id=project.id,
page_number=page_num,
png_path=str(png_path) if png_path.exists() else None,
ocr_data=page_data,
width=width,
height=height
)
print(f" [OK] Страница {page_num}: {width}x{height}")
# Загрузить QC issues
qc_path = output_folder / "dimension_qc_report.json"
if qc_path.exists():
qc = json.loads(qc_path.read_text(encoding="utf-8"))
total_imported = 0
for severity in ["errors", "warnings", "infos"]:
for item in qc.get(severity, []):
page_num = item["page"]
page = crud.get_page_by_number(db, project.id, page_num)
# Извлечь bbox
bbox = item.get("bbox") or item.get("bbox1") or item.get("bbox_dim")
x1 = y1 = x2 = y2 = None
if bbox:
if isinstance(bbox[0], list):
xs = [p[0] for p in bbox]
ys = [p[1] for p in bbox]
x1, y1, x2, y2 = min(xs), min(ys), max(xs), max(ys)
else:
x1, y1, x2, y2 = bbox[0], bbox[1], bbox[2], bbox[3]
crud.create_issue(
db,
project_id=project.id,
page_id=page.id if page else None,
issue_type=item["type"],
severity=item["severity"],
message=item["message"],
bbox_x1=x1, bbox_y1=y1, bbox_x2=x2, bbox_y2=y2,
dimension_text=item.get("text"),
confidence=item.get("confidence"),
extra_data={k: v for k, v in item.items()
if k not in ["type", "severity", "message", "page", "text", "confidence", "bbox", "bbox1", "bbox2", "bbox_dim"]}
)
total_imported += 1
print(f"[OK] Импортировано замечаний: {total_imported}")
# Загрузить VLM descriptions
vlm_path = output_folder / "vlm_descriptions.json"
if vlm_path.exists():
vlm = json.loads(vlm_path.read_text(encoding="utf-8"))
for page in project.pages:
for item in vlm.get("descriptions", []):
if item.get("page") == page.page_number:
page.vlm_description = item.get("description")
db.commit()
break
print(f"[OK] VLM описания загружены")
print(f"\n[INFO] Проект #{project.id} готов!")
print(f" Dashboard: http://localhost:8001/")
print(f" Viewer: http://localhost:8001/viewer/{project.id}/5")
print(f" Review: http://localhost:8001/review.html?project={project.id}")
return project.id
except Exception as e:
print(f"[ERR] {e}")
import traceback
traceback.print_exc()
finally:
db.close()
if __name__ == "__main__":
import_output_123()

6
backend/requirements.txt Normal file
View File

@ -0,0 +1,6 @@
fastapi>=0.104.0
uvicorn[standard]>=0.24.0
sqlalchemy>=2.0.0
pydantic>=2.5.0
python-multipart>=0.0.6
pillow>=10.0.0

329
backend/static/index.html Normal file
View File

@ -0,0 +1,329 @@
<!DOCTYPE html>
<html lang="ru">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Blueprint QC Dashboard</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: 'Segoe UI', system-ui, sans-serif;
background: #0d0d1a;
color: #eee;
min-height: 100vh;
}
.container { max-width: 1200px; margin: 0 auto; padding: 40px 20px; }
h1 { color: #e94560; margin-bottom: 30px; }
.stats {
display: grid;
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
gap: 15px;
margin-bottom: 40px;
}
.stat-card {
background: #16213e;
border: 1px solid #0f3460;
padding: 20px;
border-radius: 8px;
}
.stat-value { font-size: 32px; font-weight: bold; color: #e94560; }
.stat-label { font-size: 13px; opacity: 0.7; margin-top: 8px; }
.upload-zone {
border: 2px dashed #0f3460;
border-radius: 12px;
padding: 60px 40px;
text-align: center;
margin-bottom: 40px;
transition: all 0.3s;
}
.upload-zone:hover, .upload-zone.dragover {
border-color: #e94560;
background: rgba(233, 69, 96, 0.05);
}
.spinner {
display: inline-block;
width: 16px;
height: 16px;
border: 2px solid rgba(255,170,0,0.3);
border-top-color: #ffaa00;
border-radius: 50%;
animation: spin 1s linear infinite;
vertical-align: middle;
margin-right: 6px;
}
@keyframes spin { to { transform: rotate(360deg); } }
.processing-row { animation: pulse 2s infinite; }
@keyframes pulse { 0%, 100% { opacity: 1; } 50% { opacity: 0.7; } }
.delete-btn {
background: transparent;
border: none;
color: #ff4444;
cursor: pointer;
font-size: 16px;
margin-left: 10px;
opacity: 0.5;
}
.delete-btn:hover { opacity: 1; }
.upload-zone input { display: none; }
.upload-btn {
background: #e94560;
color: white;
border: none;
padding: 12px 32px;
border-radius: 6px;
cursor: pointer;
font-size: 16px;
margin-top: 15px;
}
.projects-table {
width: 100%;
border-collapse: collapse;
}
.projects-table th {
text-align: left;
padding: 15px;
border-bottom: 2px solid #0f3460;
font-size: 13px;
text-transform: uppercase;
opacity: 0.7;
}
.projects-table td {
padding: 15px;
border-bottom: 1px solid #1a1a2e;
}
.projects-table tr:hover { background: rgba(255,255,255,0.03); }
.status {
display: inline-block;
padding: 4px 12px;
border-radius: 20px;
font-size: 12px;
font-weight: 600;
}
.status-completed { background: rgba(0,200,100,0.2); color: #00c864; }
.status-processing { background: rgba(255,170,0,0.2); color: #ffaa00; }
.status-error { background: rgba(255,0,0,0.2); color: #ff4444; }
.status-uploaded { background: rgba(100,100,255,0.2); color: #8888ff; }
.view-link { color: #44aaff; text-decoration: none; }
.view-link:hover { text-decoration: underline; }
#progress-bar {
display: none;
width: 100%;
height: 4px;
background: #1a1a2e;
border-radius: 2px;
margin-top: 15px;
overflow: hidden;
}
#progress-fill {
width: 0%;
height: 100%;
background: #e94560;
transition: width 0.3s;
}
.error-msg { color: #ff4444; margin-top: 10px; font-size: 14px; }
</style>
</head>
<body>
<div class="container">
<h1>Blueprint QC Dashboard</h1>
<div class="stats" id="stats">
<div class="stat-card">
<div class="stat-value" id="stat-projects">-</div>
<div class="stat-label">Проектов</div>
</div>
<div class="stat-card">
<div class="stat-value" id="stat-issues">-</div>
<div class="stat-label">Замечаний</div>
</div>
<div class="stat-card">
<div class="stat-value" id="stat-feedback">-</div>
<div class="stat-label">Размечено</div>
</div>
<div class="stat-card">
<div class="stat-value" id="stat-accuracy">-</div>
<div class="stat-label">Точность</div>
</div>
</div>
<div class="upload-zone" id="uploadZone">
<h3>Перетащите PDF сюда или кликните для выбора</h3>
<p style="opacity:0.6; margin-top:10px;">Максимальный размер: 100 МБ</p>
<input type="file" id="fileInput" accept=".pdf">
<button class="upload-btn" onclick="document.getElementById('fileInput').click()">Выбрать файл</button>
<div id="progress-bar"><div id="progress-fill"></div></div>
<div id="upload-error" class="error-msg"></div>
</div>
<h2 style="margin-bottom:20px;">Проекты</h2>
<table class="projects-table" id="projectsTable">
<thead>
<tr>
<th>ID</th>
<th>Название</th>
<th>Файл</th>
<th>Статус</th>
<th>Создан</th>
<th>Замечаний</th>
<th>Действия</th>
</tr>
</thead>
<tbody></tbody>
</table>
</div>
<script>
const API = '/api';
// Stats
async function loadStats() {
try {
const res = await fetch(`${API}/stats`);
const data = await res.json();
document.getElementById('stat-projects').textContent = data.total_projects;
document.getElementById('stat-issues').textContent = data.total_issues;
document.getElementById('stat-feedback').textContent = data.feedback_stats.true_positive + data.feedback_stats.false_positive;
document.getElementById('stat-accuracy').textContent = data.accuracy_estimate ? (data.accuracy_estimate * 100).toFixed(1) + '%' : '-';
} catch (e) { console.error('Stats error:', e); }
}
// Track previous statuses for detecting completion
let prevStatuses = {};
// Projects
async function loadProjects() {
try {
const res = await fetch(`${API}/projects`);
const projects = await res.json();
const tbody = document.querySelector('#projectsTable tbody');
// Check for newly completed projects
projects.forEach(p => {
if (prevStatuses[p.id] === 'processing' && p.status === 'completed') {
showToast(`Project "${p.name || p.pdf_filename}" обработан!`, 'success');
}
prevStatuses[p.id] = p.status;
});
tbody.innerHTML = projects.map(p => {
const isProcessing = p.status === 'processing' || p.status === 'uploaded';
const statusHtml = isProcessing
? `<span class="status status-${p.status}"><span class="spinner"></span>${p.status}</span>`
: `<span class="status status-${p.status}">${p.status}</span>`;
let actions;
if (p.status === 'completed') {
actions = `<a href="/viewer/${p.id}/1" class="view-link" target="_blank">Viewer</a> | <a href="/review.html?project=${p.id}" class="view-link">Review</a> <button class="delete-btn" onclick="deleteProject(${p.id})" title="Удалить">🗑</button>`;
} else if (p.status === 'processing') {
actions = '<span style="opacity:0.5"><span class="spinner"></span> Обработка...</span>';
} else {
actions = `
<label style="font-size:11px; margin-right:6px; cursor:pointer;">
<input type="checkbox" id="tiling-${p.id}" style="vertical-align:middle;"> tiling
</label>
<button class="upload-btn" style="padding:4px 12px; font-size:12px;" onclick="analyzeProject(${p.id})">🔬 Анализировать</button>
<button class="delete-btn" onclick="deleteProject(${p.id})" title="Удалить">🗑</button>
`;
}
return `
<tr class="${isProcessing ? 'processing-row' : ''}">
<td>${p.id}</td>
<td>${p.name || '-'}</td>
<td>${p.pdf_filename}</td>
<td>${statusHtml}</td>
<td>${new Date(p.created_at).toLocaleString()}</td>
<td>${p.issues ? p.issues.length : (isProcessing ? '<span class="spinner"></span>' : '0')}</td>
<td>${actions}</td>
</tr>
`}).join('');
} catch (e) { console.error('Projects error:', e); }
}
async function deleteProject(id) {
if (!confirm('Удалить проект?')) return;
try {
const res = await fetch(`${API}/projects/${id}`, { method: 'DELETE' });
if (res.ok) {
loadProjects();
loadStats();
showToast('Проект удалён', 'success');
}
} catch(e) { showToast('Ошибка удаления', 'error'); }
}
// Upload
const uploadZone = document.getElementById('uploadZone');
const fileInput = document.getElementById('fileInput');
const progressBar = document.getElementById('progress-bar');
const progressFill = document.getElementById('progress-fill');
const uploadError = document.getElementById('upload-error');
uploadZone.addEventListener('dragover', (e) => { e.preventDefault(); uploadZone.classList.add('dragover'); });
uploadZone.addEventListener('dragleave', () => uploadZone.classList.remove('dragover'));
uploadZone.addEventListener('drop', (e) => {
e.preventDefault();
uploadZone.classList.remove('dragover');
if (e.dataTransfer.files.length) handleFile(e.dataTransfer.files[0]);
});
fileInput.addEventListener('change', (e) => { if (e.target.files.length) handleFile(e.target.files[0]); });
async function analyzeProject(id) {
try {
const useTiling = document.getElementById(`tiling-${id}`)?.checked || false;
const url = useTiling ? `${API}/projects/${id}/analyze?use_tiling=true` : `${API}/projects/${id}/analyze`;
const res = await fetch(url, { method: 'POST' });
if (res.ok) {
const data = await res.json();
showToast(`Анализ запущен (${data.ocr_engine})!`, 'success');
loadProjects();
} else {
const err = await res.text();
showToast('Ошибка запуска: ' + err, 'error');
}
} catch(e) { showToast('Ошибка сети', 'error'); }
}
async function handleFile(file) {
if (!file.name.endsWith('.pdf')) {
uploadError.textContent = 'Только PDF файлы';
return;
}
uploadError.textContent = '';
progressBar.style.display = 'block';
progressFill.style.width = '30%';
const form = new FormData();
form.append('file', file);
form.append('name', file.name);
try {
const res = await fetch(`${API}/projects/upload`, { method: 'POST', body: form });
progressFill.style.width = '100%';
if (res.ok) {
setTimeout(() => { progressBar.style.display = 'none'; progressFill.style.width = '0%'; loadProjects(); loadStats(); }, 500);
} else {
uploadError.textContent = 'Ошибка загрузки: ' + res.statusText;
progressBar.style.display = 'none';
}
} catch (e) {
uploadError.textContent = 'Ошибка сети';
progressBar.style.display = 'none';
}
}
// Toast notification
function showToast(msg, type) {
const toast = document.createElement('div');
toast.style.cssText = `position:fixed; bottom:20px; right:20px; background:rgba(0,0,0,0.95); color:white; padding:14px 24px; border-radius:8px; border:1px solid ${type==='success'?'#00c864':'#ff4444'}; z-index:100000; font-size:13px; box-shadow:0 4px 20px rgba(0,0,0,0.5);`;
toast.textContent = msg;
document.body.appendChild(toast);
setTimeout(() => toast.remove(), 4000);
}
// Init
loadStats();
loadProjects();
setInterval(() => { loadProjects(); loadStats(); }, 5000); // Обновление каждые 5 сек
</script>
</body>
</html>

495
backend/static/review.html Normal file
View File

@ -0,0 +1,495 @@
<!DOCTYPE html>
<html lang="ru">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>Review Issues — Blueprint QC</title>
<style>
* { margin: 0; padding: 0; box-sizing: border-box; }
body {
font-family: 'Segoe UI', system-ui, sans-serif;
background: #0d0d1a;
color: #eee;
min-height: 100vh;
}
.container { max-width: 1400px; margin: 0 auto; padding: 20px; }
header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 30px;
padding-bottom: 20px;
border-bottom: 2px solid #0f3460;
}
h1 { color: #e94560; }
.project-info { opacity: 0.7; font-size: 14px; }
.filters {
display: flex;
gap: 15px;
margin-bottom: 25px;
flex-wrap: wrap;
align-items: center;
}
.filter-group {
display: flex;
gap: 8px;
align-items: center;
}
.filter-group label { font-size: 13px; opacity: 0.8; }
.filter-group select, .filter-group input {
background: #1a1a2e;
border: 1px solid #0f3460;
color: #eee;
padding: 6px 12px;
border-radius: 4px;
font-size: 13px;
}
.filter-btn {
background: #e94560;
border: none;
color: white;
padding: 6px 16px;
border-radius: 4px;
cursor: pointer;
font-size: 13px;
}
.filter-btn.secondary {
background: #1a1a2e;
border: 1px solid #0f3460;
}
.progress-bar {
width: 100%;
height: 6px;
background: #1a1a2e;
border-radius: 3px;
margin-bottom: 25px;
overflow: hidden;
}
.progress-fill {
height: 100%;
background: linear-gradient(90deg, #e94560, #ffaa00);
transition: width 0.5s;
}
.progress-text {
font-size: 12px;
opacity: 0.6;
margin-bottom: 8px;
}
.issues-grid {
display: grid;
grid-template-columns: repeat(auto-fill, minmax(350px, 1fr));
gap: 15px;
}
.issue-card {
background: #16213e;
border-left: 4px solid var(--card-color, #888);
border-radius: 8px;
padding: 15px;
transition: all 0.2s;
position: relative;
}
.issue-card:hover {
background: #1a2847;
transform: translateY(-2px);
}
.issue-card.reviewed-tp { border-left-color: #00c864; }
.issue-card.reviewed-fp { opacity: 0.6; }
.issue-card.reviewed-fp .issue-text { text-decoration: line-through; }
.issue-card.reviewed-ns { border-left-color: #888; }
.issue-header {
display: flex;
justify-content: space-between;
align-items: center;
margin-bottom: 10px;
}
.issue-id {
font-size: 11px;
background: rgba(255,255,255,0.1);
padding: 2px 8px;
border-radius: 10px;
}
.issue-type {
font-size: 11px;
text-transform: uppercase;
opacity: 0.7;
}
.issue-severity {
font-size: 10px;
padding: 2px 8px;
border-radius: 10px;
font-weight: bold;
}
.severity-error { background: rgba(255,0,0,0.2); color: #ff4444; }
.severity-warning { background: rgba(255,170,0,0.2); color: #ffaa00; }
.severity-info { background: rgba(0,150,255,0.2); color: #44aaff; }
.issue-text {
font-size: 13px;
line-height: 1.5;
margin-bottom: 10px;
}
.issue-meta {
font-size: 11px;
opacity: 0.6;
margin-bottom: 12px;
}
.issue-meta span { margin-right: 15px; }
.feedback-bar {
display: flex;
gap: 8px;
}
.feedback-bar button {
flex: 1;
padding: 8px;
border: 1px solid #333;
background: #0d0d1a;
color: #ccc;
border-radius: 6px;
cursor: pointer;
font-size: 12px;
transition: all 0.2s;
}
.feedback-bar button:hover {
background: #1a1a2e;
color: #fff;
}
.feedback-bar button.tp:hover { background: rgba(0,200,100,0.2); border-color: #00c864; color: #00c864; }
.feedback-bar button.fp:hover { background: rgba(255,0,0,0.2); border-color: #ff4444; color: #ff4444; }
.feedback-bar button.ns:hover { background: rgba(150,150,150,0.2); border-color: #888; color: #888; }
.feedback-bar button:disabled { opacity: 0.5; cursor: not-allowed; }
.reviewed-badge {
position: absolute;
top: 10px;
right: 10px;
font-size: 11px;
padding: 2px 8px;
border-radius: 10px;
font-weight: bold;
}
.badge-tp { background: rgba(0,200,100,0.2); color: #00c864; }
.badge-fp { background: rgba(255,0,0,0.2); color: #ff4444; }
.badge-ns { background: rgba(150,150,150,0.2); color: #888; }
.empty-state {
text-align: center;
padding: 80px 20px;
opacity: 0.5;
}
.empty-state h2 { margin-bottom: 15px; }
.viewer-link {
color: #44aaff;
text-decoration: none;
font-size: 12px;
}
.viewer-link:hover { text-decoration: underline; }
#toast {
position: fixed;
bottom: 20px;
right: 20px;
background: rgba(0,0,0,0.95);
color: white;
padding: 14px 24px;
border-radius: 8px;
border: 1px solid #e94560;
z-index: 100000;
display: none;
font-size: 13px;
box-shadow: 0 4px 20px rgba(0,0,0,0.5);
}
.stats-row {
display: flex;
gap: 20px;
margin-bottom: 20px;
flex-wrap: wrap;
}
.stat-pill {
background: #1a1a2e;
padding: 8px 16px;
border-radius: 20px;
font-size: 13px;
}
.stat-pill .num { font-weight: bold; color: #e94560; }
</style>
</head>
<body>
<div class="container">
<header>
<div>
<h1>Review Issues</h1>
<div class="project-info" id="projectInfo">Loading...</div>
</div>
<a href="/" style="color:#44aaff; text-decoration:none;">← Dashboard</a>
</header>
<div class="stats-row" id="statsRow"></div>
<div class="progress-text" id="progressText">0% reviewed</div>
<div class="progress-bar">
<div class="progress-fill" id="progressFill" style="width:0%"></div>
</div>
<div class="filters">
<div class="filter-group">
<label>Project:</label>
<select id="projectSelect"><option value="">Select...</option></select>
</div>
<div class="filter-group">
<label>Severity:</label>
<select id="severityFilter">
<option value="all">All</option>
<option value="error">Error</option>
<option value="warning">Warning</option>
<option value="info">Info</option>
</select>
</div>
<div class="filter-group">
<label>Type:</label>
<select id="typeFilter"><option value="all">All</option></select>
</div>
<div class="filter-group">
<label>Status:</label>
<select id="statusFilter">
<option value="all">All</option>
<option value="unreviewed">Unreviewed</option>
<option value="reviewed">Reviewed</option>
</select>
</div>
<button class="filter-btn" onclick="loadIssues()">Apply</button>
<button class="filter-btn secondary" onclick="exportTrainingData()">📥 Export Training Data</button>
</div>
<div class="issues-grid" id="issuesGrid"></div>
<div class="empty-state" id="emptyState" style="display:none;">
<h2>No issues found</h2>
<p>Try adjusting filters or upload a new PDF.</p>
</div>
</div>
<div id="toast"></div>
<script>
const API = '/api';
let currentProjectId = null;
let allIssues = [];
let allProjects = [];
// Init
async function init() {
await loadProjects();
// Check URL param
const urlParams = new URLSearchParams(window.location.search);
const pid = urlParams.get('project');
if (pid) {
document.getElementById('projectSelect').value = pid;
await selectProject(pid);
}
}
async function loadProjects() {
try {
const res = await fetch(`${API}/projects`);
allProjects = await res.json();
const select = document.getElementById('projectSelect');
select.innerHTML = '<option value="">Select project...</option>' +
allProjects.map(p => `<option value="${p.id}">${p.name || p.pdf_filename} (#${p.id})</option>`).join('');
select.addEventListener('change', (e) => selectProject(e.target.value));
} catch(e) { console.error(e); }
}
async function selectProject(pid) {
if (!pid) return;
currentProjectId = pid;
const project = allProjects.find(p => p.id == pid);
if (project) {
document.getElementById('projectInfo').textContent =
`${project.name || project.pdf_filename} | Status: ${project.status}`;
}
await loadIssues();
}
async function loadIssues() {
if (!currentProjectId) {
showToast('Select a project first', 'warning');
return;
}
const severity = document.getElementById('severityFilter').value;
const typeFilter = document.getElementById('typeFilter').value;
const statusFilter = document.getElementById('statusFilter').value;
try {
let url = `${API}/projects/${currentProjectId}/issues`;
const params = [];
if (severity !== 'all') params.push(`severity=${severity}`);
if (typeFilter !== 'all') params.push(`issue_type=${typeFilter}`);
if (statusFilter === 'reviewed') params.push(`has_feedback=true`);
if (statusFilter === 'unreviewed') params.push(`has_feedback=false`);
if (params.length) url += '?' + params.join('&');
const res = await fetch(url);
allIssues = await res.json();
renderIssues();
updateStats();
updateTypeFilter();
} catch(e) { showToast('Error loading issues: ' + e.message, 'error'); }
}
function renderIssues() {
const grid = document.getElementById('issuesGrid');
const empty = document.getElementById('emptyState');
if (!allIssues.length) {
grid.innerHTML = '';
empty.style.display = 'block';
return;
}
empty.style.display = 'none';
grid.innerHTML = allIssues.map(issue => {
const color = issue.severity === 'error' ? '#ff4444' : (issue.severity === 'warning' ? '#ffaa00' : '#44aaff');
const reviewedClass = issue.feedback ?
(issue.feedback.is_true_positive === true ? 'reviewed-tp' :
(issue.feedback.is_true_positive === false ? 'reviewed-fp' : 'reviewed-ns')) : '';
const badge = issue.feedback ?
(issue.feedback.is_true_positive === true ? '<span class="reviewed-badge badge-tp">✅ TP</span>' :
(issue.feedback.is_true_positive === false ? '<span class="reviewed-badge badge-fp">❌ FP</span>' :
'<span class="reviewed-badge badge-ns">🤷 NS</span>')) : '';
return `
<div class="issue-card ${reviewedClass}" style="--card-color:${color}" data-issue-id="${issue.id}">
${badge}
<div class="issue-header">
<span class="issue-id">#${issue.id}</span>
<span class="issue-severity severity-${issue.severity}">${issue.severity}</span>
</div>
<div class="issue-type">${issue.issue_type}</div>
<div class="issue-text">${issue.message}</div>
<div class="issue-meta">
<span>Page ${issue.page_number || '?'}</span>
<span>Conf: ${issue.confidence ? issue.confidence.toFixed(2) : 'N/A'}</span>
${issue.dimension_text ? `<span>Dim: "${issue.dimension_text}"</span>` : ''}
<a href="/viewer/${issue.project_id}/${issue.page_number || 1}" class="viewer-link" target="_blank">Open Viewer →</a>
</div>
<div class="feedback-bar">
<button class="tp" onclick="submitFeedback(${issue.id}, true, this)" ${issue.feedback ? 'disabled' : ''}>✅ Real Issue</button>
<button class="fp" onclick="submitFeedback(${issue.id}, false, this)" ${issue.feedback ? 'disabled' : ''}>❌ False Positive</button>
<button class="ns" onclick="submitFeedback(${issue.id}, null, this)" ${issue.feedback ? 'disabled' : ''}>🤷 Not Sure</button>
</div>
</div>
`;
}).join('');
}
async function submitFeedback(issueId, isTP, btn) {
const card = btn.closest('.issue-card');
const btns = card.querySelectorAll('.feedback-bar button');
btns.forEach(b => b.disabled = true);
btn.textContent = '...';
try {
const res = await fetch(`${API}/feedback`, {
method: 'POST',
headers: {'Content-Type': 'application/json'},
body: JSON.stringify({
issue_id: issueId,
is_true_positive: isTP,
action_taken: isTP === true ? 'fixed' : (isTP === false ? 'ignored' : 'not_sure')
})
});
if (res.ok) {
// Update the issue in memory
const issue = allIssues.find(i => i.id === issueId);
if (issue) {
issue.feedback = { is_true_positive: isTP };
}
renderIssues();
updateStats();
showToast('Saved!', 'success');
} else {
const err = await res.text();
showToast('Error: ' + err, 'error');
btns.forEach(b => b.disabled = false);
btn.textContent = isTP === true ? '✅ Real Issue' : (isTP === false ? '❌ False Positive' : '🤷 Not Sure');
}
} catch(e) {
showToast('Network error: ' + e.message, 'error');
btns.forEach(b => b.disabled = false);
btn.textContent = isTP === true ? '✅ Real Issue' : (isTP === false ? '❌ False Positive' : '🤷 Not Sure');
}
}
function updateStats() {
const total = allIssues.length;
const reviewed = allIssues.filter(i => i.feedback).length;
const tp = allIssues.filter(i => i.feedback && i.feedback.is_true_positive === true).length;
const fp = allIssues.filter(i => i.feedback && i.feedback.is_true_positive === false).length;
const pct = total ? Math.round((reviewed / total) * 100) : 0;
document.getElementById('progressFill').style.width = pct + '%';
document.getElementById('progressText').textContent =
`${pct}% reviewed (${reviewed}/${total}) | ✅ TP: ${tp} | ❌ FP: ${fp}`;
document.getElementById('statsRow').innerHTML = `
<div class="stat-pill">Total: <span class="num">${total}</span></div>
<div class="stat-pill">Reviewed: <span class="num">${reviewed}</span></div>
<div class="stat-pill">True Positive: <span class="num" style="color:#00c864">${tp}</span></div>
<div class="stat-pill">False Positive: <span class="num" style="color:#ff4444">${fp}</span></div>
`;
}
function updateTypeFilter() {
const types = [...new Set(allIssues.map(i => i.issue_type))];
const select = document.getElementById('typeFilter');
const current = select.value;
select.innerHTML = '<option value="all">All</option>' +
types.map(t => `<option value="${t}">${t}</option>`).join('');
if (types.includes(current)) select.value = current;
}
async function exportTrainingData() {
if (!currentProjectId) {
showToast('Select a project first', 'warning');
return;
}
try {
const res = await fetch(`${API}/training/export?project_id=${currentProjectId}&only_labeled=true`);
const data = await res.json();
// Download as file
const blob = new Blob([JSON.stringify(data, null, 2)], {type: 'application/json'});
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = `training_data_project_${currentProjectId}_${new Date().toISOString().slice(0,10)}.json`;
a.click();
URL.revokeObjectURL(url);
showToast(`Exported ${data.total_samples} samples!`, 'success');
} catch(e) {
showToast('Export error: ' + e.message, 'error');
}
}
function showToast(msg, type) {
const toast = document.getElementById('toast');
toast.textContent = msg;
toast.style.borderColor = type === 'success' ? '#00c864' : (type === 'error' ? '#ff4444' : '#ffaa00');
toast.style.display = 'block';
setTimeout(() => toast.style.display = 'none', 3000);
}
init();
</script>
</body>
</html>