Add FastAPI backend with DZI viewer and feedback system
- FastAPI app with SQLite DB (projects, pages, issues, feedback) - OpenSeadragon DZI viewer with inline SVG overlays - Dashboard: upload, project list, tiling toggle, review mode - Pipeline integration: tiling OCR → layout → elements → rules QC → DZI → DB - Feedback collection: true_positive / false_positive / not_sure per issue
This commit is contained in:
Кирилл Блинов
parent
feeb02242b
commit
f37c477a0a
122
backend/README.md
Normal file
122
backend/README.md
Normal file
@ -0,0 +1,122 @@
|
||||
# Blueprint QC Backend
|
||||
|
||||
FastAPI backend для сбора, хранения и разметки замечаний QC по чертежам.
|
||||
|
||||
## Архитектура
|
||||
|
||||
```
|
||||
┌─────────────┐ ┌──────────────┐ ┌─────────────┐
|
||||
│ React UI │────▶│ FastAPI │────▶│ SQLite │
|
||||
│ (viewer) │◀────│ Backend │◀────│ Database │
|
||||
└─────────────┘ └──────────────┘ └─────────────┘
|
||||
│
|
||||
▼
|
||||
┌──────────────┐
|
||||
│ OCR + QC │
|
||||
│ Pipeline │
|
||||
└──────────────┘
|
||||
```
|
||||
|
||||
## Установка
|
||||
|
||||
```bash
|
||||
cd backend
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## Запуск
|
||||
|
||||
```bash
|
||||
python -m uvicorn app.main:app --reload --port 8000
|
||||
```
|
||||
|
||||
API будет доступен на http://localhost:8000
|
||||
|
||||
Документация (Swagger): http://localhost:8000/docs
|
||||
|
||||
## API Endpoints
|
||||
|
||||
### Проекты
|
||||
|
||||
| Method | Endpoint | Описание |
|
||||
|--------|----------|----------|
|
||||
| POST | `/api/projects/upload` | Загрузка PDF, запуск обработки |
|
||||
| GET | `/api/projects` | Список проектов |
|
||||
| GET | `/api/projects/{id}` | Детали проекта |
|
||||
|
||||
### Замечания (Issues)
|
||||
|
||||
| Method | Endpoint | Описание |
|
||||
|--------|----------|----------|
|
||||
| GET | `/api/projects/{id}/issues` | Замечания проекта |
|
||||
| GET | `/api/issues/{id}` | Одно замечание |
|
||||
|
||||
### Feedback (разметка)
|
||||
|
||||
| Method | Endpoint | Описание |
|
||||
|--------|----------|----------|
|
||||
| POST | `/api/feedback` | Отметить TP/FP |
|
||||
| GET | `/api/feedback/stats` | Статистика |
|
||||
|
||||
### Обучение
|
||||
|
||||
| Method | Endpoint | Описание |
|
||||
|--------|----------|----------|
|
||||
| GET | `/api/training/data` | Данные для обучения |
|
||||
| POST | `/api/training/export` | Скачать JSON |
|
||||
| GET | `/api/stats` | Общая статистика |
|
||||
|
||||
### Viewer
|
||||
|
||||
| Method | Endpoint | Описание |
|
||||
|--------|----------|----------|
|
||||
| GET | `/viewer/{project_id}/{page}` | HTML viewer |
|
||||
| GET | `/viewer_tiles/{project_id}/{path}` | DZI тайлы |
|
||||
|
||||
## Workflow
|
||||
|
||||
1. **Загрузка**: `POST /api/projects/upload` с PDF
|
||||
2. **Обработка**: Backend запускает OCR → QC → DZI (фоново)
|
||||
3. **Просмотр**: `GET /viewer/{id}/{page}` — viewer с overlay
|
||||
4. **Разметка**: Пользователь кликает замечания → `POST /api/feedback` (is_true_positive: true/false)
|
||||
5. **Обучение**: `GET /api/training/data` — экспорт размеченных данных
|
||||
|
||||
## Feedback Schema
|
||||
|
||||
```json
|
||||
{
|
||||
"issue_id": 123,
|
||||
"is_true_positive": true, // true = реальная проблема, false = ложное срабатывание
|
||||
"comment": "Размер действительно плохо читается",
|
||||
"action_taken": "fixed" // fixed / ignored / not_sure
|
||||
}
|
||||
```
|
||||
|
||||
## Training Data
|
||||
|
||||
Каждый размеченный пример содержит:
|
||||
- `bbox`: координаты на PNG
|
||||
- `issue_type`: тип проблемы
|
||||
- `is_true_positive`: метка от пользователя
|
||||
- `image_path`: путь к PNG страницы
|
||||
- `dimension_text`: текст размера (если есть)
|
||||
- `confidence`: OCR confidence
|
||||
|
||||
Накопив 100-200 размеченных примеров, можно:
|
||||
1. Fine-tune VLM (few-shot prompting)
|
||||
2. Обучить YOLO-детектор под ваши типы чертежей
|
||||
3. Дообучить правила QC (эвристики)
|
||||
|
||||
## Модели БД
|
||||
|
||||
### Project
|
||||
- id, name, pdf_filename, status, created_at, output_folder
|
||||
|
||||
### Page
|
||||
- id, project_id, page_number, png_path, ocr_data, vlm_description
|
||||
|
||||
### Issue
|
||||
- id, project_id, page_id, issue_type, severity, message, bbox, dimension_text, confidence
|
||||
|
||||
### Feedback
|
||||
- id, issue_id, is_true_positive, comment, action_taken, created_at
|
||||
1
backend/__init__.py
Normal file
1
backend/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
# backend/__init__.py
|
||||
1
backend/app/__init__.py
Normal file
1
backend/app/__init__.py
Normal file
@ -0,0 +1 @@
|
||||
# app/__init__.py
|
||||
192
backend/app/crud.py
Normal file
192
backend/app/crud.py
Normal file
@ -0,0 +1,192 @@
|
||||
# app/crud.py
|
||||
from sqlalchemy.orm import Session, joinedload
|
||||
from sqlalchemy import func
|
||||
from typing import List, Optional
|
||||
import app.models as models
|
||||
import app.schemas as schemas
|
||||
|
||||
|
||||
# ---------- Project ----------
|
||||
def create_project(db: Session, pdf_filename: str, name: Optional[str] = None) -> models.Project:
|
||||
project = models.Project(
|
||||
name=name or pdf_filename,
|
||||
pdf_filename=pdf_filename,
|
||||
status="uploaded"
|
||||
)
|
||||
db.add(project)
|
||||
db.commit()
|
||||
db.refresh(project)
|
||||
return project
|
||||
|
||||
def get_project(db: Session, project_id: int) -> Optional[models.Project]:
|
||||
return db.query(models.Project).options(
|
||||
joinedload(models.Project.pages),
|
||||
joinedload(models.Project.issues).joinedload(models.Issue.feedback)
|
||||
).filter(models.Project.id == project_id).first()
|
||||
|
||||
def get_projects(db: Session, skip: int = 0, limit: int = 100) -> List[models.Project]:
|
||||
return db.query(models.Project).options(
|
||||
joinedload(models.Project.pages),
|
||||
joinedload(models.Project.issues).joinedload(models.Issue.feedback)
|
||||
).order_by(models.Project.created_at.desc()).offset(skip).limit(limit).all()
|
||||
|
||||
def update_project_status(db: Session, project_id: int, status: str, error_message: str = None, output_folder: str = None):
|
||||
project = get_project(db, project_id)
|
||||
if project:
|
||||
project.status = status
|
||||
if error_message is not None:
|
||||
project.error_message = error_message
|
||||
# Очищаем ошибку при новом запуске или успехе
|
||||
if status in ("processing", "completed") and error_message is None:
|
||||
project.error_message = None
|
||||
if output_folder:
|
||||
project.output_folder = output_folder
|
||||
if status == "completed":
|
||||
from datetime import datetime
|
||||
project.completed_at = datetime.utcnow()
|
||||
db.commit()
|
||||
db.refresh(project)
|
||||
return project
|
||||
|
||||
|
||||
# ---------- Page ----------
|
||||
def create_page(db: Session, project_id: int, page_number: int, **kwargs) -> models.Page:
|
||||
page = models.Page(project_id=project_id, page_number=page_number, **kwargs)
|
||||
db.add(page)
|
||||
db.commit()
|
||||
db.refresh(page)
|
||||
return page
|
||||
|
||||
def get_page_by_number(db: Session, project_id: int, page_number: int) -> Optional[models.Page]:
|
||||
return db.query(models.Page).filter(
|
||||
models.Page.project_id == project_id,
|
||||
models.Page.page_number == page_number
|
||||
).first()
|
||||
|
||||
|
||||
# ---------- Issue ----------
|
||||
def create_issue(db: Session, project_id: int, page_id: Optional[int], **kwargs) -> models.Issue:
|
||||
issue = models.Issue(project_id=project_id, page_id=page_id, **kwargs)
|
||||
db.add(issue)
|
||||
db.commit()
|
||||
db.refresh(issue)
|
||||
return issue
|
||||
|
||||
def get_issues(db: Session, project_id: Optional[int] = None, page_id: Optional[int] = None,
|
||||
severity: Optional[str] = None, issue_type: Optional[str] = None,
|
||||
has_feedback: Optional[bool] = None, skip: int = 0, limit: int = 1000):
|
||||
query = db.query(models.Issue)
|
||||
if project_id:
|
||||
query = query.filter(models.Issue.project_id == project_id)
|
||||
if page_id:
|
||||
query = query.filter(models.Issue.page_id == page_id)
|
||||
if severity:
|
||||
query = query.filter(models.Issue.severity == severity)
|
||||
if issue_type:
|
||||
query = query.filter(models.Issue.issue_type == issue_type)
|
||||
if has_feedback is not None:
|
||||
if has_feedback:
|
||||
query = query.filter(models.Issue.feedback != None)
|
||||
else:
|
||||
query = query.filter(models.Issue.feedback == None)
|
||||
return query.order_by(models.Issue.created_at.desc()).offset(skip).limit(limit).all()
|
||||
|
||||
def get_issue(db: Session, issue_id: int) -> Optional[models.Issue]:
|
||||
return db.query(models.Issue).filter(models.Issue.id == issue_id).first()
|
||||
|
||||
|
||||
# ---------- Feedback ----------
|
||||
def create_feedback(db: Session, feedback: schemas.FeedbackCreate) -> models.Feedback:
|
||||
# Удалить старый feedback если есть
|
||||
existing = db.query(models.Feedback).filter(models.Feedback.issue_id == feedback.issue_id).first()
|
||||
if existing:
|
||||
db.delete(existing)
|
||||
db.commit()
|
||||
|
||||
db_feedback = models.Feedback(**feedback.dict())
|
||||
db.add(db_feedback)
|
||||
db.commit()
|
||||
db.refresh(db_feedback)
|
||||
return db_feedback
|
||||
|
||||
def get_feedback_stats(db: Session, project_id: Optional[int] = None):
|
||||
query = db.query(models.Feedback)
|
||||
if project_id:
|
||||
query = query.join(models.Issue).filter(models.Issue.project_id == project_id)
|
||||
|
||||
total = query.count()
|
||||
true_positive = query.filter(models.Feedback.is_true_positive == True).count()
|
||||
false_positive = query.filter(models.Feedback.is_true_positive == False).count()
|
||||
unreviewed = query.filter(models.Feedback.is_true_positive == None).count()
|
||||
|
||||
accuracy = true_positive / (true_positive + false_positive) if (true_positive + false_positive) > 0 else None
|
||||
|
||||
return {
|
||||
"total": total,
|
||||
"true_positive": true_positive,
|
||||
"false_positive": false_positive,
|
||||
"unreviewed": unreviewed,
|
||||
"accuracy_estimate": round(accuracy, 3) if accuracy else None
|
||||
}
|
||||
|
||||
|
||||
# ---------- Stats ----------
|
||||
def get_stats(db: Session) -> schemas.StatsResponse:
|
||||
total_projects = db.query(models.Project).count()
|
||||
total_issues = db.query(models.Issue).count()
|
||||
|
||||
# По типам
|
||||
issue_types = db.query(models.Issue.issue_type, func.count(models.Issue.id)).group_by(models.Issue.issue_type).all()
|
||||
issues_by_type = {t[0]: t[1] for t in issue_types}
|
||||
|
||||
# Feedback
|
||||
fb_stats = get_feedback_stats(db)
|
||||
|
||||
return schemas.StatsResponse(
|
||||
total_projects=total_projects,
|
||||
total_issues=total_issues,
|
||||
issues_by_type=issues_by_type,
|
||||
feedback_stats={
|
||||
"true_positive": fb_stats["true_positive"],
|
||||
"false_positive": fb_stats["false_positive"],
|
||||
"unreviewed": fb_stats["unreviewed"]
|
||||
},
|
||||
accuracy_estimate=fb_stats["accuracy_estimate"]
|
||||
)
|
||||
|
||||
|
||||
# ---------- Training Data ----------
|
||||
def export_training_data(db: Session, project_id: Optional[int] = None,
|
||||
only_labeled: bool = True) -> List[schemas.TrainingSample]:
|
||||
"""Экспорт данных для обучения ML-модели."""
|
||||
query = db.query(models.Issue)
|
||||
if project_id:
|
||||
query = query.filter(models.Issue.project_id == project_id)
|
||||
if only_labeled:
|
||||
query = query.join(models.Feedback)
|
||||
|
||||
issues = query.all()
|
||||
samples = []
|
||||
|
||||
for issue in issues:
|
||||
sample = schemas.TrainingSample(
|
||||
issue_id=issue.id,
|
||||
issue_type=issue.issue_type,
|
||||
severity=issue.severity,
|
||||
message=issue.message,
|
||||
bbox={
|
||||
"x1": issue.bbox_x1,
|
||||
"y1": issue.bbox_y1,
|
||||
"x2": issue.bbox_x2,
|
||||
"y2": issue.bbox_y2
|
||||
},
|
||||
dimension_text=issue.dimension_text,
|
||||
confidence=issue.confidence,
|
||||
page_number=issue.page.page_number if issue.page else None,
|
||||
is_true_positive=issue.feedback.is_true_positive if issue.feedback else None,
|
||||
image_path=issue.page.png_path if issue.page else None,
|
||||
label=f"{'good' if issue.feedback and issue.feedback.is_true_positive else 'bad'}_{issue.issue_type.lower()}" if issue.feedback else None
|
||||
)
|
||||
samples.append(sample)
|
||||
|
||||
return samples
|
||||
23
backend/app/database.py
Normal file
23
backend/app/database.py
Normal file
@ -0,0 +1,23 @@
|
||||
# app/database.py
|
||||
import os
|
||||
from sqlalchemy import create_engine
|
||||
from sqlalchemy.ext.declarative import declarative_base
|
||||
from sqlalchemy.orm import sessionmaker
|
||||
|
||||
SQLALCHEMY_DATABASE_URL = os.getenv("DATABASE_URL", "sqlite:///./blueprint_qc.db")
|
||||
|
||||
engine = create_engine(
|
||||
SQLALCHEMY_DATABASE_URL,
|
||||
connect_args={"check_same_thread": False} if SQLALCHEMY_DATABASE_URL.startswith("sqlite") else {}
|
||||
)
|
||||
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
|
||||
|
||||
Base = declarative_base()
|
||||
|
||||
|
||||
def get_db():
|
||||
db = SessionLocal()
|
||||
try:
|
||||
yield db
|
||||
finally:
|
||||
db.close()
|
||||
401
backend/app/main.py
Normal file
401
backend/app/main.py
Normal file
@ -0,0 +1,401 @@
|
||||
# app/main.py
|
||||
"""
|
||||
FastAPI backend для Blueprint QC Service.
|
||||
|
||||
API Endpoints:
|
||||
- POST /api/projects/upload — загрузка PDF
|
||||
- GET /api/projects — список проектов
|
||||
- GET /api/projects/{id} — детали проекта
|
||||
- GET /api/projects/{id}/issues — замечания проекта
|
||||
- POST /api/feedback — отметить замечание (TP/FP)
|
||||
- GET /api/feedback/stats — статистика feedback
|
||||
- GET /api/training/data — экспорт данных для обучения
|
||||
- GET /api/stats — общая статистика
|
||||
- GET /viewer/{project_id}/{page} — HTML viewer (серверный, без file://)
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import re
|
||||
import json
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import List, Optional
|
||||
from datetime import datetime
|
||||
|
||||
from fastapi import FastAPI, File, UploadFile, Depends, HTTPException, Query
|
||||
from fastapi.responses import HTMLResponse, FileResponse, JSONResponse
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.staticfiles import StaticFiles
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
# Добавить пути
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
|
||||
sys.path.insert(0, str(Path(__file__).parent))
|
||||
|
||||
from app.database import engine, get_db, Base
|
||||
from app import models, schemas, crud, processing
|
||||
|
||||
# Создать таблицы
|
||||
Base.metadata.create_all(bind=engine)
|
||||
|
||||
# Создать папку static если нет
|
||||
STATIC_DIR = Path(__file__).parent.parent / "static"
|
||||
STATIC_DIR.mkdir(exist_ok=True)
|
||||
|
||||
app = FastAPI(
|
||||
title="Blueprint QC API",
|
||||
description="Сервис автоматической проверки чертежей и сбора данных для обучения",
|
||||
version="0.1.0"
|
||||
)
|
||||
|
||||
# CORS
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["*"],
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
|
||||
# Пути
|
||||
# Абсолютные пути (backend запускается из разных директорий)
|
||||
BASE_DIR = Path(__file__).parent.parent # backend/
|
||||
UPLOAD_DIR = BASE_DIR / "uploads"
|
||||
OUTPUT_DIR = BASE_DIR / "outputs"
|
||||
UPLOAD_DIR.mkdir(exist_ok=True)
|
||||
OUTPUT_DIR.mkdir(exist_ok=True)
|
||||
|
||||
|
||||
# ==================== PROJECTS ====================
|
||||
|
||||
@app.post("/api/projects/upload", response_model=schemas.ProjectResponse)
|
||||
async def upload_pdf(
|
||||
file: UploadFile = File(...),
|
||||
name: Optional[str] = None,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
"""Загрузка PDF файла. Запускает фоновую обработку."""
|
||||
if not file.filename.endswith('.pdf'):
|
||||
raise HTTPException(400, "Only PDF files allowed")
|
||||
|
||||
# Сохранить файл
|
||||
safe_name = Path(file.filename).name
|
||||
pdf_path = UPLOAD_DIR / f"{datetime.now().strftime('%Y%m%d_%H%M%S')}_{safe_name}"
|
||||
with open(pdf_path, "wb") as f:
|
||||
shutil.copyfileobj(file.file, f)
|
||||
|
||||
# Создать проект в БД (статус = uploaded, анализ запускается вручную)
|
||||
project = crud.create_project(db, pdf_filename=safe_name, name=name or safe_name)
|
||||
|
||||
# НЕ запускаем обработку автоматически — пользователь жмет "Анализировать"
|
||||
# вручную, когда готов
|
||||
|
||||
return project
|
||||
|
||||
|
||||
@app.get("/api/projects", response_model=List[schemas.ProjectDetail])
|
||||
def list_projects(skip: int = 0, limit: int = 100, db: Session = Depends(get_db)):
|
||||
"""Список всех проектов со страницами и замечаниями."""
|
||||
return crud.get_projects(db, skip=skip, limit=limit)
|
||||
|
||||
|
||||
@app.get("/api/projects/{project_id}", response_model=schemas.ProjectDetail)
|
||||
def get_project(project_id: int, db: Session = Depends(get_db)):
|
||||
"""Детали проекта со страницами и замечаниями."""
|
||||
project = crud.get_project(db, project_id)
|
||||
if not project:
|
||||
raise HTTPException(404, "Project not found")
|
||||
return project
|
||||
|
||||
|
||||
@app.delete("/api/projects/{project_id}")
|
||||
def delete_project(project_id: int, db: Session = Depends(get_db)):
|
||||
"""Удалить проект и все связанные данные."""
|
||||
project = crud.get_project(db, project_id)
|
||||
if not project:
|
||||
raise HTTPException(404, "Project not found")
|
||||
|
||||
# Удалить файлы проекта
|
||||
if project.output_folder:
|
||||
import shutil
|
||||
try:
|
||||
shutil.rmtree(project.output_folder, ignore_errors=True)
|
||||
except:
|
||||
pass
|
||||
|
||||
db.delete(project)
|
||||
db.commit()
|
||||
return {"status": "deleted", "project_id": project_id}
|
||||
|
||||
|
||||
@app.post("/api/projects/{project_id}/analyze")
|
||||
def analyze_project(project_id: int, use_tiling: bool = False, db: Session = Depends(get_db)):
|
||||
"""Запустить анализ (OCR + Layout + QC + DZI) для проекта."""
|
||||
project = crud.get_project(db, project_id)
|
||||
if not project:
|
||||
raise HTTPException(404, "Project not found")
|
||||
|
||||
if project.status == "processing":
|
||||
return {"status": "already_processing", "project_id": project_id}
|
||||
|
||||
if project.status == "completed":
|
||||
pass
|
||||
|
||||
pdf_files = list(UPLOAD_DIR.glob(f"*_{project.pdf_filename}"))
|
||||
if not pdf_files:
|
||||
pdf_files = list(UPLOAD_DIR.glob(project.pdf_filename))
|
||||
|
||||
if not pdf_files:
|
||||
raise HTTPException(404, f"PDF file not found in {UPLOAD_DIR}")
|
||||
|
||||
pdf_path = pdf_files[0]
|
||||
|
||||
crud.update_project_status(db, project_id, "processing")
|
||||
|
||||
import threading
|
||||
def run_in_background():
|
||||
db_local = next(get_db())
|
||||
try:
|
||||
processing.run_pipeline(project.id, pdf_path, OUTPUT_DIR, db_local, use_tiling=use_tiling)
|
||||
except Exception as e:
|
||||
print(f"[ERROR] Pipeline failed for project {project.id}: {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
crud.update_project_status(db_local, project.id, "error", error_message=str(e))
|
||||
finally:
|
||||
db_local.close()
|
||||
|
||||
thread = threading.Thread(target=run_in_background)
|
||||
thread.start()
|
||||
|
||||
return {"status": "processing_started", "project_id": project_id, "ocr_engine": "tiling" if use_tiling else "standard"}
|
||||
|
||||
|
||||
# ==================== ISSUES ====================
|
||||
|
||||
@app.get("/api/projects/{project_id}/issues")
|
||||
def get_issues(
|
||||
project_id: int,
|
||||
severity: Optional[str] = Query(None, description="error / warning / info"),
|
||||
issue_type: Optional[str] = Query(None),
|
||||
has_feedback: Optional[bool] = Query(None, description="Filter by feedback presence"),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
"""Получить замечания проекта."""
|
||||
issues = crud.get_issues(
|
||||
db, project_id=project_id, severity=severity,
|
||||
issue_type=issue_type, has_feedback=has_feedback
|
||||
)
|
||||
|
||||
# Build response with page_number manually
|
||||
result = []
|
||||
for issue in issues:
|
||||
item = {
|
||||
"id": issue.id,
|
||||
"project_id": issue.project_id,
|
||||
"page_id": issue.page_id,
|
||||
"page_number": issue.page.page_number if issue.page else None,
|
||||
"issue_type": issue.issue_type,
|
||||
"severity": issue.severity,
|
||||
"message": issue.message,
|
||||
"bbox_x1": issue.bbox_x1,
|
||||
"bbox_y1": issue.bbox_y1,
|
||||
"bbox_x2": issue.bbox_x2,
|
||||
"bbox_y2": issue.bbox_y2,
|
||||
"dimension_text": issue.dimension_text,
|
||||
"confidence": issue.confidence,
|
||||
"created_at": issue.created_at.isoformat() if issue.created_at else None,
|
||||
"feedback": {
|
||||
"id": issue.feedback.id,
|
||||
"issue_id": issue.feedback.issue_id,
|
||||
"is_true_positive": issue.feedback.is_true_positive,
|
||||
"comment": issue.feedback.comment,
|
||||
"action_taken": issue.feedback.action_taken,
|
||||
"created_at": issue.feedback.created_at.isoformat() if issue.feedback.created_at else None
|
||||
} if issue.feedback else None
|
||||
}
|
||||
result.append(item)
|
||||
|
||||
return result
|
||||
|
||||
|
||||
@app.get("/api/issues/{issue_id}", response_model=schemas.IssueResponse)
|
||||
def get_issue(issue_id: int, db: Session = Depends(get_db)):
|
||||
"""Одно замечание."""
|
||||
issue = crud.get_issue(db, issue_id)
|
||||
if not issue:
|
||||
raise HTTPException(404, "Issue not found")
|
||||
return issue
|
||||
|
||||
|
||||
# ==================== FEEDBACK ====================
|
||||
|
||||
@app.post("/api/feedback", response_model=schemas.FeedbackResponse)
|
||||
def submit_feedback(feedback: schemas.FeedbackCreate, db: Session = Depends(get_db)):
|
||||
"""
|
||||
Отправить feedback по замечанию.
|
||||
|
||||
is_true_positive:
|
||||
- true = реальная проблема (правильное срабатывание)
|
||||
- false = ложное срабатывание (false positive)
|
||||
- null = не уверен
|
||||
|
||||
action_taken: fixed / ignored / not_sure
|
||||
"""
|
||||
issue = crud.get_issue(db, feedback.issue_id)
|
||||
if not issue:
|
||||
raise HTTPException(404, "Issue not found")
|
||||
|
||||
return crud.create_feedback(db, feedback)
|
||||
|
||||
|
||||
@app.get("/api/feedback/stats")
|
||||
def feedback_stats(project_id: Optional[int] = None, db: Session = Depends(get_db)):
|
||||
"""Статистика feedback."""
|
||||
return crud.get_feedback_stats(db, project_id=project_id)
|
||||
|
||||
|
||||
# ==================== VIEWER ====================
|
||||
|
||||
@app.get("/viewer/{project_id}/{page_number}", response_class=HTMLResponse)
|
||||
def get_viewer(project_id: int, page_number: int, db: Session = Depends(get_db)):
|
||||
"""HTML viewer с overlay замечаний + feedback buttons."""
|
||||
project = crud.get_project(db, project_id)
|
||||
if not project or not project.output_folder:
|
||||
raise HTTPException(404, "Project or output not found")
|
||||
|
||||
# ВСЕГДА перегенерировать viewer для нужной страницы
|
||||
# (иначе тайлы указывают на другую страницу)
|
||||
result = processing.generate_viewer_html(db, project_id, page_number)
|
||||
if not result:
|
||||
raise HTTPException(404, "Viewer not available")
|
||||
|
||||
viewer_path = Path(result)
|
||||
content = viewer_path.read_text(encoding="utf-8")
|
||||
|
||||
# Подменить пути тайлов — теперь они будут правильные
|
||||
# (generate_web_viewer.py уже сгенерировал для page_{page_number:03d}_files/)
|
||||
content = content.replace(
|
||||
'Url: "./page_',
|
||||
f'Url: "/viewer_tiles/{project_id}/page_'
|
||||
)
|
||||
|
||||
# Внедрить project_id и page info для навигации
|
||||
total_pages = len(project.pages) if project.pages else page_number
|
||||
content = content.replace('const PROJECT_ID = null;', f'const PROJECT_ID = {project_id};')
|
||||
content = content.replace('const TOTAL_PAGES = null;', f'const TOTAL_PAGES = {total_pages};')
|
||||
content = content.replace('const CURRENT_PAGE = null;', f'const CURRENT_PAGE = {page_number};')
|
||||
|
||||
# Обновить счётчик страниц в навбаре
|
||||
content = content.replace(
|
||||
f'<span id="totalPages">?</span>',
|
||||
f'<span id="totalPages">{total_pages}</span>'
|
||||
)
|
||||
|
||||
# Получить issue IDs из БД для этой страницы
|
||||
page = crud.get_page_by_number(db, project_id, page_number)
|
||||
if page:
|
||||
issues = crud.get_issues(db, project_id=project_id, page_id=page.id)
|
||||
issue_db_ids = [str(i.id) for i in issues]
|
||||
|
||||
# Внедрить DB IDs в HTML
|
||||
content = content.replace('data-has-api="false"', 'data-has-api="true"')
|
||||
|
||||
def inject_db_id(match):
|
||||
idx = int(match.group(1)) - 1
|
||||
if idx < len(issue_db_ids):
|
||||
return match.group(0) + f' data-db-id="{issue_db_ids[idx]}"'
|
||||
return match.group(0)
|
||||
|
||||
content = re.sub(r'data-id="(\d+)"', inject_db_id, content)
|
||||
|
||||
return HTMLResponse(content=content)
|
||||
|
||||
|
||||
@app.get("/viewer_tiles/{project_id}/{filename:path}")
|
||||
def get_tile(project_id: int, filename: str, db: Session = Depends(get_db)):
|
||||
"""Отдаёт DZI тайлы."""
|
||||
project = crud.get_project(db, project_id)
|
||||
if not project or not project.output_folder:
|
||||
raise HTTPException(404)
|
||||
|
||||
tile_path = Path(project.output_folder) / filename
|
||||
if tile_path.exists() and str(tile_path).startswith(str(project.output_folder)):
|
||||
return FileResponse(tile_path)
|
||||
raise HTTPException(404)
|
||||
|
||||
|
||||
# ==================== TRAINING DATA ====================
|
||||
|
||||
@app.get("/api/training/data", response_model=schemas.TrainingDataExport)
|
||||
def export_training_data(
|
||||
project_id: Optional[int] = Query(None),
|
||||
only_labeled: bool = Query(True),
|
||||
format: str = Query("json", description="json / yolo / csv"),
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
"""
|
||||
Экспорт данных для обучения ML-модели.
|
||||
|
||||
only_labeled=true — только размеченные feedback'ом замечания.
|
||||
"""
|
||||
samples = crud.export_training_data(db, project_id=project_id, only_labeled=only_labeled)
|
||||
|
||||
if format == "json":
|
||||
return schemas.TrainingDataExport(
|
||||
total_samples=len(samples),
|
||||
labeled_samples=len([s for s in samples if s.is_true_positive is not None]),
|
||||
samples=samples,
|
||||
export_format="json"
|
||||
)
|
||||
|
||||
# TODO: YOLO / CSV форматы
|
||||
raise HTTPException(400, f"Format {format} not yet implemented")
|
||||
|
||||
|
||||
@app.post("/api/training/export")
|
||||
def download_training_export(
|
||||
project_id: Optional[int] = None,
|
||||
only_labeled: bool = True,
|
||||
db: Session = Depends(get_db)
|
||||
):
|
||||
"""Скачать training data как JSON файл."""
|
||||
samples = crud.export_training_data(db, project_id=project_id, only_labeled=only_labeled)
|
||||
|
||||
export = {
|
||||
"generated_at": datetime.utcnow().isoformat(),
|
||||
"total_samples": len(samples),
|
||||
"samples": [s.dict() for s in samples]
|
||||
}
|
||||
|
||||
return JSONResponse(content=export, media_type="application/json")
|
||||
|
||||
|
||||
# ==================== STATS ====================
|
||||
|
||||
@app.get("/api/stats", response_model=schemas.StatsResponse)
|
||||
def get_stats(db: Session = Depends(get_db)):
|
||||
"""Общая статистика системы."""
|
||||
return crud.get_stats(db)
|
||||
|
||||
|
||||
# ==================== HEALTH ====================
|
||||
|
||||
@app.get("/api/health")
|
||||
def health_check():
|
||||
return {"status": "ok", "version": "0.1.0"}
|
||||
|
||||
|
||||
# ==================== STATIC (для демо) ====================
|
||||
|
||||
# Подключить статику
|
||||
try:
|
||||
app.mount("/", StaticFiles(directory=str(STATIC_DIR), html=True), name="static")
|
||||
except RuntimeError:
|
||||
pass # Already mounted
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import uvicorn
|
||||
uvicorn.run(app, host="0.0.0.0", port=8000)
|
||||
95
backend/app/models.py
Normal file
95
backend/app/models.py
Normal file
@ -0,0 +1,95 @@
|
||||
# app/models.py
|
||||
from sqlalchemy import Column, Integer, String, Float, DateTime, Text, ForeignKey, JSON, Boolean
|
||||
from sqlalchemy.orm import relationship
|
||||
from datetime import datetime
|
||||
from app.database import Base
|
||||
|
||||
|
||||
class Project(Base):
|
||||
"""Проект — один загруженный PDF."""
|
||||
__tablename__ = "projects"
|
||||
|
||||
id = Column(Integer, primary_key=True, index=True)
|
||||
name = Column(String, index=True)
|
||||
pdf_filename = Column(String, nullable=False)
|
||||
status = Column(String, default="uploaded") # uploaded / processing / completed / error
|
||||
created_at = Column(DateTime, default=datetime.utcnow)
|
||||
completed_at = Column(DateTime, nullable=True)
|
||||
error_message = Column(Text, nullable=True)
|
||||
|
||||
# Пути к файлам
|
||||
output_folder = Column(String, nullable=True)
|
||||
|
||||
pages = relationship("Page", back_populates="project", cascade="all, delete-orphan")
|
||||
issues = relationship("Issue", back_populates="project", cascade="all, delete-orphan")
|
||||
|
||||
|
||||
class Page(Base):
|
||||
"""Страница PDF."""
|
||||
__tablename__ = "pages"
|
||||
|
||||
id = Column(Integer, primary_key=True, index=True)
|
||||
project_id = Column(Integer, ForeignKey("projects.id"))
|
||||
page_number = Column(Integer, nullable=False)
|
||||
png_path = Column(String, nullable=True)
|
||||
dzi_path = Column(String, nullable=True)
|
||||
ocr_data = Column(JSON, nullable=True) # full_ocr_results для этой страницы
|
||||
vlm_description = Column(Text, nullable=True)
|
||||
width = Column(Integer, nullable=True)
|
||||
height = Column(Integer, nullable=True)
|
||||
|
||||
project = relationship("Project", back_populates="pages")
|
||||
issues = relationship("Issue", back_populates="page")
|
||||
|
||||
|
||||
class Issue(Base):
|
||||
"""Замечание QC — одна проблема на чертеже."""
|
||||
__tablename__ = "issues"
|
||||
|
||||
id = Column(Integer, primary_key=True, index=True)
|
||||
project_id = Column(Integer, ForeignKey("projects.id"))
|
||||
page_id = Column(Integer, ForeignKey("pages.id"))
|
||||
|
||||
issue_type = Column(String, nullable=False) # DIMENSION_OVERLAP, LOW_CONFIDENCE, etc.
|
||||
severity = Column(String, nullable=False) # error / warning / info
|
||||
message = Column(Text, nullable=False)
|
||||
|
||||
# Координаты bbox на PNG (в пикселях)
|
||||
bbox_x1 = Column(Float, nullable=True)
|
||||
bbox_y1 = Column(Float, nullable=True)
|
||||
bbox_x2 = Column(Float, nullable=True)
|
||||
bbox_y2 = Column(Float, nullable=True)
|
||||
|
||||
# Дополнительные данные
|
||||
dimension_text = Column(String, nullable=True) # Текст размера (если применимо)
|
||||
confidence = Column(Float, nullable=True) # OCR confidence
|
||||
extra_data = Column(JSON, nullable=True) # Всё остальное
|
||||
source = Column(String, nullable=True) # "rules" или "vlm"
|
||||
|
||||
created_at = Column(DateTime, default=datetime.utcnow)
|
||||
|
||||
project = relationship("Project", back_populates="issues")
|
||||
page = relationship("Page", back_populates="issues")
|
||||
feedback = relationship("Feedback", back_populates="issue", uselist=False)
|
||||
|
||||
|
||||
class Feedback(Base):
|
||||
"""Feedback проектировщика — правда ли это замечание."""
|
||||
__tablename__ = "feedback"
|
||||
|
||||
id = Column(Integer, primary_key=True, index=True)
|
||||
issue_id = Column(Integer, ForeignKey("issues.id"), unique=True)
|
||||
|
||||
# True = реальная проблема, False = ложное срабатывание, None = не размечено
|
||||
is_true_positive = Column(Boolean, nullable=True)
|
||||
|
||||
# Почему (опционально)
|
||||
comment = Column(Text, nullable=True)
|
||||
|
||||
# Что сделал проектировщик
|
||||
action_taken = Column(String, nullable=True) # fixed / ignored / not_sure
|
||||
|
||||
created_at = Column(DateTime, default=datetime.utcnow)
|
||||
user_id = Column(String, nullable=True) # Для многопользовательского режима
|
||||
|
||||
issue = relationship("Issue", back_populates="feedback")
|
||||
217
backend/app/processing.py
Normal file
217
backend/app/processing.py
Normal file
@ -0,0 +1,217 @@
|
||||
# app/processing.py
|
||||
"""
|
||||
Интеграция с существующими скриптами OCR, QC, DZI.
|
||||
Запускает pipeline в фоне и сохраняет результаты в БД.
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
import json
|
||||
import re
|
||||
import subprocess
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
from sqlalchemy.orm import Session
|
||||
|
||||
# Добавить корень проекта в path для импорта скриптов
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent))
|
||||
|
||||
import app.crud as crud
|
||||
import app.models as models
|
||||
|
||||
|
||||
def run_pipeline(project_id: int, pdf_path: Path, output_base: Path, db: Session, use_tiling: bool = False):
|
||||
"""Запускает полный pipeline: PDF → OCR → Layout → Elements → QC → DZI → БД."""
|
||||
|
||||
project = crud.get_project(db, project_id)
|
||||
if not project:
|
||||
return
|
||||
|
||||
crud.update_project_status(db, project_id, "processing")
|
||||
|
||||
try:
|
||||
# 1. Создать output папку
|
||||
output_folder = output_base / f"project_{project_id}"
|
||||
output_folder.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
script_dir = Path(__file__).parent.parent.parent # backend/app -> backend -> opencode
|
||||
|
||||
# 2. OCR + PNG (RapidOCR или Tiling OCR)
|
||||
cmd = [sys.executable, str(script_dir / "process_any_pdf.py"), str(pdf_path), str(output_folder)]
|
||||
if use_tiling:
|
||||
cmd.append("--use-tiling")
|
||||
print("[INFO] Tiling OCR enabled")
|
||||
_run_command(cmd, cwd=str(script_dir))
|
||||
|
||||
# 3. Layout Detection для каждой оригинальной страницы
|
||||
# Только page_NNN.png, исключая визуализации (_dims, _layout и т.д.)
|
||||
page_pngs = sorted([p for p in output_folder.glob("page_*.png")
|
||||
if not any(suffix in p.stem for suffix in ["_dims", "_layout", "_detected", "_preproc", "_ocr_compare"])])
|
||||
for png in page_pngs:
|
||||
ocr_json = output_folder / "full_ocr_results.json"
|
||||
if ocr_json.exists():
|
||||
try:
|
||||
_run_command([
|
||||
sys.executable, str(script_dir / "layout_detector.py"),
|
||||
str(png), str(ocr_json)
|
||||
], cwd=str(script_dir))
|
||||
print(f"[INFO] Layout detection done for {png.name}")
|
||||
except Exception as e:
|
||||
print(f"[WARN] Layout detection failed for {png.name}: {e}")
|
||||
|
||||
# 4. Multi-Element Extraction (dimensions, positions, GOSTs, etc.)
|
||||
for png in page_pngs:
|
||||
ocr_json = output_folder / "full_ocr_results.json"
|
||||
layout_json = output_folder / "layout.json"
|
||||
if ocr_json.exists() and layout_json.exists():
|
||||
try:
|
||||
_run_command([
|
||||
sys.executable, str(script_dir / "multi_element_extractor.py"),
|
||||
str(png), str(ocr_json), str(layout_json)
|
||||
], cwd=str(script_dir))
|
||||
print(f"[INFO] Element extraction done for {png.name}")
|
||||
except Exception as e:
|
||||
print(f"[WARN] Element extraction failed for {png.name}: {e}")
|
||||
|
||||
# 5. QC (dimension_qc_checker.py) — правила
|
||||
_run_command([
|
||||
sys.executable, str(script_dir / "dimension_qc_checker.py"),
|
||||
str(output_folder)
|
||||
], cwd=str(script_dir))
|
||||
|
||||
# 6. DZI для каждой страницы
|
||||
for png in page_pngs:
|
||||
_run_command([
|
||||
sys.executable, str(script_dir / "generate_dzi.py"),
|
||||
str(png)
|
||||
], cwd=str(script_dir))
|
||||
|
||||
# 7. Проверить результаты
|
||||
ocr_path = output_folder / "full_ocr_results.json"
|
||||
if not ocr_path.exists():
|
||||
raise RuntimeError(f"OCR results not generated: {ocr_path}")
|
||||
|
||||
# 8. Загрузить в БД
|
||||
_import_results(db, project_id, output_folder)
|
||||
|
||||
crud.update_project_status(db, project_id, "completed", output_folder=str(output_folder))
|
||||
|
||||
except Exception as e:
|
||||
crud.update_project_status(db, project_id, "error", error_message=str(e))
|
||||
raise
|
||||
|
||||
|
||||
def _run_command(cmd: list, cwd: Optional[Path] = None):
|
||||
"""Запускает команду, проверяет exit code, выбрасывает исключение при ошибке."""
|
||||
result = subprocess.run(
|
||||
cmd,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
cwd=cwd
|
||||
)
|
||||
if result.returncode != 0:
|
||||
stderr = result.stderr[:1000]
|
||||
stdout = result.stdout[:500]
|
||||
raise RuntimeError(f"Command failed ({result.returncode}): {' '.join(cmd)}\nSTDERR: {stderr}\nSTDOUT: {stdout}")
|
||||
return result
|
||||
|
||||
|
||||
def _import_results(db: Session, project_id: int, output_folder: Path):
|
||||
"""Импорт OCR и QC результатов в БД. Очищает старые данные проекта перед импортом."""
|
||||
|
||||
# Очистить старые страницы и замечания (cascade удалит issues и feedback)
|
||||
db.query(models.Page).filter(models.Page.project_id == project_id).delete(synchronize_session=False)
|
||||
db.query(models.Issue).filter(models.Issue.project_id == project_id).delete(synchronize_session=False)
|
||||
db.commit()
|
||||
|
||||
# Загрузить OCR
|
||||
ocr_path = output_folder / "full_ocr_results.json"
|
||||
if ocr_path.exists():
|
||||
ocr = json.loads(ocr_path.read_text(encoding="utf-8"))
|
||||
for page_data in ocr.get("pages", []):
|
||||
page_num = page_data["page_number"]
|
||||
png_path = output_folder / f"page_{page_num:03d}.png"
|
||||
|
||||
page = crud.create_page(
|
||||
db, project_id=project_id, page_number=page_num,
|
||||
png_path=str(png_path) if png_path.exists() else None,
|
||||
ocr_data=page_data
|
||||
)
|
||||
|
||||
# Загрузить VLM extraction descriptions в pages
|
||||
vlm_path = output_folder / "vlm_extraction.json"
|
||||
vlm_data = {}
|
||||
if vlm_path.exists():
|
||||
vlm_data = json.loads(vlm_path.read_text(encoding="utf-8"))
|
||||
|
||||
# Обновить vlm_description для каждой страницы
|
||||
for img_name, extraction in vlm_data.items():
|
||||
page_num_match = re.search(r"page_(\d+)", img_name)
|
||||
if page_num_match:
|
||||
page_num = int(page_num_match.group(1))
|
||||
page = crud.get_page_by_number(db, project_id, page_num)
|
||||
if page:
|
||||
desc = extraction.get("description", "")
|
||||
page.vlm_description = desc[:2000] if desc else None # ограничим длину
|
||||
db.commit()
|
||||
|
||||
# Загрузить QC issues (только rules — VLM issues удалены как ненадёжные)
|
||||
qc_path = output_folder / "dimension_qc_report.json"
|
||||
if qc_path.exists():
|
||||
qc = json.loads(qc_path.read_text(encoding="utf-8"))
|
||||
|
||||
for severity in ["errors", "warnings", "infos"]:
|
||||
for item in qc.get(severity, []):
|
||||
page_num = item.get("page")
|
||||
if not page_num:
|
||||
continue
|
||||
page = crud.get_page_by_number(db, project_id, page_num)
|
||||
|
||||
# Извлечь bbox
|
||||
bbox = item.get("bbox") or item.get("bbox1") or item.get("bbox_dim")
|
||||
x1 = y1 = x2 = y2 = None
|
||||
if bbox:
|
||||
if isinstance(bbox[0], list):
|
||||
xs = [p[0] for p in bbox]
|
||||
ys = [p[1] for p in bbox]
|
||||
x1, y1, x2, y2 = min(xs), min(ys), max(xs), max(ys)
|
||||
else:
|
||||
x1, y1, x2, y2 = bbox[0], bbox[1], bbox[2], bbox[3]
|
||||
|
||||
crud.create_issue(
|
||||
db, project_id=project_id, page_id=page.id if page else None,
|
||||
issue_type=item.get("type", "UNKNOWN"),
|
||||
severity=item.get("severity", "warning"),
|
||||
message=item.get("message", ""),
|
||||
bbox_x1=x1, bbox_y1=y1, bbox_x2=x2, bbox_y2=y2,
|
||||
dimension_text=item.get("text"),
|
||||
confidence=item.get("confidence"),
|
||||
source="rules",
|
||||
extra_data={k: v for k, v in item.items() if k not in ["type", "severity", "message", "page", "text", "confidence", "bbox", "bbox1", "bbox2", "bbox_dim", "source"]}
|
||||
)
|
||||
|
||||
|
||||
def generate_viewer_html(db: Session, project_id: int, page_number: int) -> Optional[str]:
|
||||
"""Генерирует HTML viewer для конкретной страницы."""
|
||||
project = crud.get_project(db, project_id)
|
||||
if not project or not project.output_folder:
|
||||
return None
|
||||
|
||||
output_folder = Path(project.output_folder)
|
||||
|
||||
# Перегенерировать viewer для нужной страницы
|
||||
# Запускаем из папки, где находится generate_web_viewer.py
|
||||
script_dir = Path(__file__).parent.parent.parent # backend/app -> backend -> opencode
|
||||
result = _run_command([
|
||||
sys.executable, str(script_dir / "generate_web_viewer.py"), str(output_folder), str(page_number)
|
||||
], cwd=str(script_dir))
|
||||
|
||||
if result.returncode != 0:
|
||||
print(f"[ERROR] generate_web_viewer.py failed: {result.stderr[:500]}")
|
||||
return None
|
||||
|
||||
viewer_path = output_folder / "web_viewer" / "index.html"
|
||||
if viewer_path.exists():
|
||||
return str(viewer_path)
|
||||
return None
|
||||
137
backend/app/schemas.py
Normal file
137
backend/app/schemas.py
Normal file
@ -0,0 +1,137 @@
|
||||
# app/schemas.py
|
||||
from pydantic import BaseModel
|
||||
from typing import List, Optional, Dict, Any
|
||||
from datetime import datetime
|
||||
|
||||
|
||||
# ---------- Project ----------
|
||||
class ProjectBase(BaseModel):
|
||||
name: Optional[str] = None
|
||||
|
||||
class ProjectCreate(ProjectBase):
|
||||
pdf_filename: str
|
||||
|
||||
class ProjectResponse(ProjectBase):
|
||||
id: int
|
||||
pdf_filename: str
|
||||
status: str
|
||||
created_at: datetime
|
||||
completed_at: Optional[datetime] = None
|
||||
error_message: Optional[str] = None
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
|
||||
class ProjectDetail(ProjectResponse):
|
||||
pages: List["PageResponse"] = []
|
||||
issues: List["IssueResponse"] = []
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
|
||||
|
||||
# ---------- Page ----------
|
||||
class PageBase(BaseModel):
|
||||
page_number: int
|
||||
width: Optional[int] = None
|
||||
height: Optional[int] = None
|
||||
|
||||
class PageResponse(PageBase):
|
||||
id: int
|
||||
png_path: Optional[str] = None
|
||||
vlm_description: Optional[str] = None
|
||||
ocr_data: Optional[Dict[str, Any]] = None
|
||||
issue_count: int = 0
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
|
||||
|
||||
# ---------- Issue ----------
|
||||
class IssueBase(BaseModel):
|
||||
issue_type: str
|
||||
severity: str
|
||||
message: str
|
||||
page_number: Optional[int] = None
|
||||
|
||||
class IssueCreate(IssueBase):
|
||||
bbox_x1: Optional[float] = None
|
||||
bbox_y1: Optional[float] = None
|
||||
bbox_x2: Optional[float] = None
|
||||
bbox_y2: Optional[float] = None
|
||||
dimension_text: Optional[str] = None
|
||||
confidence: Optional[float] = None
|
||||
extra_data: Optional[Dict[str, Any]] = None
|
||||
|
||||
class IssueResponse(IssueBase):
|
||||
id: int
|
||||
project_id: int
|
||||
bbox_x1: Optional[float] = None
|
||||
bbox_y1: Optional[float] = None
|
||||
bbox_x2: Optional[float] = None
|
||||
bbox_y2: Optional[float] = None
|
||||
dimension_text: Optional[str] = None
|
||||
confidence: Optional[float] = None
|
||||
source: Optional[str] = None
|
||||
created_at: datetime
|
||||
feedback: Optional["FeedbackResponse"] = None
|
||||
page_id: Optional[int] = None
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
|
||||
|
||||
# ---------- Feedback ----------
|
||||
class FeedbackCreate(BaseModel):
|
||||
issue_id: int
|
||||
is_true_positive: Optional[bool] = None
|
||||
comment: Optional[str] = None
|
||||
action_taken: Optional[str] = None # fixed / ignored / not_sure
|
||||
|
||||
class FeedbackResponse(BaseModel):
|
||||
id: int
|
||||
issue_id: int
|
||||
is_true_positive: Optional[bool] = None
|
||||
comment: Optional[str] = None
|
||||
action_taken: Optional[str] = None
|
||||
created_at: datetime
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
|
||||
|
||||
# ---------- Stats ----------
|
||||
class StatsResponse(BaseModel):
|
||||
total_projects: int
|
||||
total_issues: int
|
||||
issues_by_type: Dict[str, int]
|
||||
feedback_stats: Dict[str, int] # true_positive, false_positive, unreviewed
|
||||
accuracy_estimate: Optional[float] = None # Доля правильных срабатываний
|
||||
|
||||
|
||||
# ---------- Training Data ----------
|
||||
class TrainingSample(BaseModel):
|
||||
"""Один пример для обучения ML-модели."""
|
||||
issue_id: int
|
||||
issue_type: str
|
||||
severity: str
|
||||
message: str
|
||||
bbox: Dict[str, float] # x1, y1, x2, y2
|
||||
dimension_text: Optional[str] = None
|
||||
confidence: Optional[float] = None
|
||||
page_number: int
|
||||
is_true_positive: Optional[bool] = None
|
||||
|
||||
# Для обучения детектора (YOLO)
|
||||
image_path: Optional[str] = None # Путь к PNG страницы
|
||||
crop_path: Optional[str] = None # Вырезанный фрагмент bbox
|
||||
label: Optional[str] = None # "good_dimension" / "bad_placement" / etc.
|
||||
|
||||
class TrainingDataExport(BaseModel):
|
||||
total_samples: int
|
||||
labeled_samples: int # Есть is_true_positive
|
||||
samples: List[TrainingSample]
|
||||
export_format: str # json / yolo / csv
|
||||
|
||||
class Config:
|
||||
from_attributes = True
|
||||
223
backend/backend.log
Normal file
223
backend/backend.log
Normal file
@ -0,0 +1,223 @@
|
||||
INFO: Started server process [49915]
|
||||
INFO: Waiting for application startup.
|
||||
INFO: Application startup complete.
|
||||
INFO: Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)
|
||||
INFO: 127.0.0.1:59330 - "GET /api/health HTTP/1.1" 200 OK
|
||||
[WARN] Command failed: /opt/homebrew/opt/python@3.11/bin/python3.11 generate_web_viewer.py /Users/kirillblinov/development/opencode/OCR/opencode/output_123 5
|
||||
stderr: /opt/homebrew/Cellar/python@3.11/3.11.15_1/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python: can't open file '/Users/kirillblinov/development/opencode/OCR/generate_web_viewer.py': [Errno 2] No such file or directory
|
||||
|
||||
INFO: 127.0.0.1:59332 - "GET /viewer/1/5 HTTP/1.1" 500 Internal Server Error
|
||||
ERROR: Exception in ASGI application
|
||||
Traceback (most recent call last):
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/protocols/http/httptools_impl.py", line 421, in run_asgi
|
||||
result = await app( # type: ignore[func-returns-value]
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/middleware/proxy_headers.py", line 56, in __call__
|
||||
return await self.app(scope, receive, send)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/applications.py", line 1159, in __call__
|
||||
await super().__call__(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/applications.py", line 90, in __call__
|
||||
await self.middleware_stack(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 186, in __call__
|
||||
raise exc
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 164, in __call__
|
||||
await self.app(scope, receive, _send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/cors.py", line 88, in __call__
|
||||
await self.app(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
|
||||
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
|
||||
raise exc
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
|
||||
await app(scope, receive, sender)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
|
||||
await self.app(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 660, in __call__
|
||||
await self.middleware_stack(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 680, in app
|
||||
await route.handle(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 276, in handle
|
||||
await self.app(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 134, in app
|
||||
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
|
||||
raise exc
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
|
||||
await app(scope, receive, sender)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 120, in app
|
||||
response = await f(request)
|
||||
^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 674, in app
|
||||
raw_response = await run_endpoint_function(
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 330, in run_endpoint_function
|
||||
return await run_in_threadpool(dependant.call, **values)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/concurrency.py", line 32, in run_in_threadpool
|
||||
return await anyio.to_thread.run_sync(func)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/to_thread.py", line 63, in run_sync
|
||||
return await get_async_backend().run_sync_in_worker_thread(
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 2518, in run_sync_in_worker_thread
|
||||
return await future
|
||||
^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 1002, in run
|
||||
result = context.run(func, *args)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/development/opencode/OCR/opencode/backend/app/main.py", line 285, in get_viewer
|
||||
content = re.sub(r'data-id="(\d+)"', inject_db_id, content)
|
||||
^^
|
||||
NameError: name 're' is not defined
|
||||
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/14/33_5.png HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/14/33_4.png HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_2.png HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_1.png HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_3.png HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_0.png HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59355 - "GET /viewer_tiles/1/page_005_files/13/17_4.png HTTP/1.1" 200 OK
|
||||
[WARN] Command failed: /opt/homebrew/opt/python@3.11/bin/python3.11 generate_web_viewer.py /Users/kirillblinov/development/opencode/OCR/opencode/output_123 5
|
||||
stderr: /opt/homebrew/Cellar/python@3.11/3.11.15_1/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python: can't open file '/Users/kirillblinov/development/opencode/OCR/generate_web_viewer.py': [Errno 2] No such file or directory
|
||||
|
||||
INFO: 127.0.0.1:59372 - "GET /viewer/1/5 HTTP/1.1" 500 Internal Server Error
|
||||
ERROR: Exception in ASGI application
|
||||
Traceback (most recent call last):
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/protocols/http/httptools_impl.py", line 421, in run_asgi
|
||||
result = await app( # type: ignore[func-returns-value]
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/middleware/proxy_headers.py", line 56, in __call__
|
||||
return await self.app(scope, receive, send)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/applications.py", line 1159, in __call__
|
||||
await super().__call__(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/applications.py", line 90, in __call__
|
||||
await self.middleware_stack(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 186, in __call__
|
||||
raise exc
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 164, in __call__
|
||||
await self.app(scope, receive, _send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/cors.py", line 88, in __call__
|
||||
await self.app(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
|
||||
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
|
||||
raise exc
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
|
||||
await app(scope, receive, sender)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
|
||||
await self.app(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 660, in __call__
|
||||
await self.middleware_stack(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 680, in app
|
||||
await route.handle(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 276, in handle
|
||||
await self.app(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 134, in app
|
||||
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
|
||||
raise exc
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
|
||||
await app(scope, receive, sender)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 120, in app
|
||||
response = await f(request)
|
||||
^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 674, in app
|
||||
raw_response = await run_endpoint_function(
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 330, in run_endpoint_function
|
||||
return await run_in_threadpool(dependant.call, **values)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/concurrency.py", line 32, in run_in_threadpool
|
||||
return await anyio.to_thread.run_sync(func)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/to_thread.py", line 63, in run_sync
|
||||
return await get_async_backend().run_sync_in_worker_thread(
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 2518, in run_sync_in_worker_thread
|
||||
return await future
|
||||
^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 1002, in run
|
||||
result = context.run(func, *args)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/development/opencode/OCR/opencode/backend/app/main.py", line 285, in get_viewer
|
||||
content = re.sub(r'data-id="(\d+)"', inject_db_id, content)
|
||||
^^
|
||||
NameError: name 're' is not defined
|
||||
INFO: 127.0.0.1:59355 - "GET /api/projects HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59406 - "GET /api/stats HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59355 - "GET /api/projects HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59408 - "GET /api/stats HTTP/1.1" 200 OK
|
||||
[WARN] Command failed: /opt/homebrew/opt/python@3.11/bin/python3.11 generate_web_viewer.py /Users/kirillblinov/development/opencode/OCR/opencode/output_123 5
|
||||
stderr: /opt/homebrew/Cellar/python@3.11/3.11.15_1/Frameworks/Python.framework/Versions/3.11/Resources/Python.app/Contents/MacOS/Python: can't open file '/Users/kirillblinov/development/opencode/OCR/generate_web_viewer.py': [Errno 2] No such file or directory
|
||||
|
||||
INFO: 127.0.0.1:59425 - "GET /viewer/1/5 HTTP/1.1" 500 Internal Server Error
|
||||
ERROR: Exception in ASGI application
|
||||
Traceback (most recent call last):
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/protocols/http/httptools_impl.py", line 421, in run_asgi
|
||||
result = await app( # type: ignore[func-returns-value]
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/uvicorn/middleware/proxy_headers.py", line 56, in __call__
|
||||
return await self.app(scope, receive, send)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/applications.py", line 1159, in __call__
|
||||
await super().__call__(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/applications.py", line 90, in __call__
|
||||
await self.middleware_stack(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 186, in __call__
|
||||
raise exc
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/errors.py", line 164, in __call__
|
||||
await self.app(scope, receive, _send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/cors.py", line 88, in __call__
|
||||
await self.app(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/middleware/exceptions.py", line 63, in __call__
|
||||
await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
|
||||
raise exc
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
|
||||
await app(scope, receive, sender)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/middleware/asyncexitstack.py", line 18, in __call__
|
||||
await self.app(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 660, in __call__
|
||||
await self.middleware_stack(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 680, in app
|
||||
await route.handle(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/routing.py", line 276, in handle
|
||||
await self.app(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 134, in app
|
||||
await wrap_app_handling_exceptions(app, request)(scope, receive, send)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
|
||||
raise exc
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
|
||||
await app(scope, receive, sender)
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 120, in app
|
||||
response = await f(request)
|
||||
^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 674, in app
|
||||
raw_response = await run_endpoint_function(
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/fastapi/routing.py", line 330, in run_endpoint_function
|
||||
return await run_in_threadpool(dependant.call, **values)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/starlette/concurrency.py", line 32, in run_in_threadpool
|
||||
return await anyio.to_thread.run_sync(func)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/to_thread.py", line 63, in run_sync
|
||||
return await get_async_backend().run_sync_in_worker_thread(
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 2518, in run_sync_in_worker_thread
|
||||
return await future
|
||||
^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/Library/Python/3.11/lib/python/site-packages/anyio/_backends/_asyncio.py", line 1002, in run
|
||||
result = context.run(func, *args)
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
File "/Users/kirillblinov/development/opencode/OCR/opencode/backend/app/main.py", line 285, in get_viewer
|
||||
content = re.sub(r'data-id="(\d+)"', inject_db_id, content)
|
||||
^^
|
||||
NameError: name 're' is not defined
|
||||
INFO: 127.0.0.1:59553 - "GET /viewer_tiles/1/page_005_files/14/18_1.png HTTP/1.1" 200 OK
|
||||
INFO: 127.0.0.1:59553 - "GET /viewer_tiles/1/page_005_files/14/18_0.png HTTP/1.1" 200 OK
|
||||
INFO: Shutting down
|
||||
INFO: Waiting for application shutdown.
|
||||
INFO: Application shutdown complete.
|
||||
INFO: Finished server process [49915]
|
||||
137
backend/import_existing.py
Normal file
137
backend/import_existing.py
Normal file
@ -0,0 +1,137 @@
|
||||
#!/usr/bin/env python3
|
||||
# -*- coding: utf-8 -*-
|
||||
"""
|
||||
Импорт существующего output_123 в backend БД.
|
||||
|
||||
Использование:
|
||||
python import_existing.py
|
||||
"""
|
||||
|
||||
import sys
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
# Добавить backend в path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent / "backend"))
|
||||
|
||||
from app.database import SessionLocal, Base, engine
|
||||
from app import models, crud, schemas
|
||||
|
||||
|
||||
def import_output_123():
|
||||
"""Импорт существующего проекта output_123 в БД."""
|
||||
|
||||
output_folder = Path("/Users/kirillblinov/development/opencode/OCR/opencode/output_123")
|
||||
if not output_folder.exists():
|
||||
print(f"[ERR] Не найден: {output_folder}")
|
||||
return
|
||||
|
||||
db = SessionLocal()
|
||||
|
||||
try:
|
||||
# Создать проект
|
||||
project = crud.create_project(
|
||||
db,
|
||||
pdf_filename="123.pdf",
|
||||
name="Test Project 123"
|
||||
)
|
||||
print(f"[INFO] Создан проект: ID={project.id}")
|
||||
|
||||
# Обновить статус
|
||||
crud.update_project_status(db, project.id, "completed", output_folder=str(output_folder))
|
||||
|
||||
# Загрузить OCR
|
||||
ocr_path = output_folder / "full_ocr_results.json"
|
||||
if ocr_path.exists():
|
||||
ocr = json.loads(ocr_path.read_text(encoding="utf-8"))
|
||||
|
||||
# Создать страницы
|
||||
for page_data in ocr.get("pages", []):
|
||||
page_num = page_data["page_number"]
|
||||
png_path = output_folder / f"page_{page_num:03d}.png"
|
||||
|
||||
from PIL import Image
|
||||
width = height = None
|
||||
if png_path.exists():
|
||||
with Image.open(png_path) as img:
|
||||
width, height = img.size
|
||||
|
||||
page = crud.create_page(
|
||||
db,
|
||||
project_id=project.id,
|
||||
page_number=page_num,
|
||||
png_path=str(png_path) if png_path.exists() else None,
|
||||
ocr_data=page_data,
|
||||
width=width,
|
||||
height=height
|
||||
)
|
||||
print(f" [OK] Страница {page_num}: {width}x{height}")
|
||||
|
||||
# Загрузить QC issues
|
||||
qc_path = output_folder / "dimension_qc_report.json"
|
||||
if qc_path.exists():
|
||||
qc = json.loads(qc_path.read_text(encoding="utf-8"))
|
||||
|
||||
total_imported = 0
|
||||
for severity in ["errors", "warnings", "infos"]:
|
||||
for item in qc.get(severity, []):
|
||||
page_num = item["page"]
|
||||
page = crud.get_page_by_number(db, project.id, page_num)
|
||||
|
||||
# Извлечь bbox
|
||||
bbox = item.get("bbox") or item.get("bbox1") or item.get("bbox_dim")
|
||||
x1 = y1 = x2 = y2 = None
|
||||
if bbox:
|
||||
if isinstance(bbox[0], list):
|
||||
xs = [p[0] for p in bbox]
|
||||
ys = [p[1] for p in bbox]
|
||||
x1, y1, x2, y2 = min(xs), min(ys), max(xs), max(ys)
|
||||
else:
|
||||
x1, y1, x2, y2 = bbox[0], bbox[1], bbox[2], bbox[3]
|
||||
|
||||
crud.create_issue(
|
||||
db,
|
||||
project_id=project.id,
|
||||
page_id=page.id if page else None,
|
||||
issue_type=item["type"],
|
||||
severity=item["severity"],
|
||||
message=item["message"],
|
||||
bbox_x1=x1, bbox_y1=y1, bbox_x2=x2, bbox_y2=y2,
|
||||
dimension_text=item.get("text"),
|
||||
confidence=item.get("confidence"),
|
||||
extra_data={k: v for k, v in item.items()
|
||||
if k not in ["type", "severity", "message", "page", "text", "confidence", "bbox", "bbox1", "bbox2", "bbox_dim"]}
|
||||
)
|
||||
total_imported += 1
|
||||
|
||||
print(f"[OK] Импортировано замечаний: {total_imported}")
|
||||
|
||||
# Загрузить VLM descriptions
|
||||
vlm_path = output_folder / "vlm_descriptions.json"
|
||||
if vlm_path.exists():
|
||||
vlm = json.loads(vlm_path.read_text(encoding="utf-8"))
|
||||
for page in project.pages:
|
||||
for item in vlm.get("descriptions", []):
|
||||
if item.get("page") == page.page_number:
|
||||
page.vlm_description = item.get("description")
|
||||
db.commit()
|
||||
break
|
||||
print(f"[OK] VLM описания загружены")
|
||||
|
||||
print(f"\n[INFO] Проект #{project.id} готов!")
|
||||
print(f" Dashboard: http://localhost:8001/")
|
||||
print(f" Viewer: http://localhost:8001/viewer/{project.id}/5")
|
||||
print(f" Review: http://localhost:8001/review.html?project={project.id}")
|
||||
|
||||
return project.id
|
||||
|
||||
except Exception as e:
|
||||
print(f"[ERR] {e}")
|
||||
import traceback
|
||||
traceback.print_exc()
|
||||
finally:
|
||||
db.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
import_output_123()
|
||||
6
backend/requirements.txt
Normal file
6
backend/requirements.txt
Normal file
@ -0,0 +1,6 @@
|
||||
fastapi>=0.104.0
|
||||
uvicorn[standard]>=0.24.0
|
||||
sqlalchemy>=2.0.0
|
||||
pydantic>=2.5.0
|
||||
python-multipart>=0.0.6
|
||||
pillow>=10.0.0
|
||||
329
backend/static/index.html
Normal file
329
backend/static/index.html
Normal file
@ -0,0 +1,329 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="ru">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Blueprint QC Dashboard</title>
|
||||
<style>
|
||||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||||
body {
|
||||
font-family: 'Segoe UI', system-ui, sans-serif;
|
||||
background: #0d0d1a;
|
||||
color: #eee;
|
||||
min-height: 100vh;
|
||||
}
|
||||
.container { max-width: 1200px; margin: 0 auto; padding: 40px 20px; }
|
||||
h1 { color: #e94560; margin-bottom: 30px; }
|
||||
.stats {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fit, minmax(200px, 1fr));
|
||||
gap: 15px;
|
||||
margin-bottom: 40px;
|
||||
}
|
||||
.stat-card {
|
||||
background: #16213e;
|
||||
border: 1px solid #0f3460;
|
||||
padding: 20px;
|
||||
border-radius: 8px;
|
||||
}
|
||||
.stat-value { font-size: 32px; font-weight: bold; color: #e94560; }
|
||||
.stat-label { font-size: 13px; opacity: 0.7; margin-top: 8px; }
|
||||
.upload-zone {
|
||||
border: 2px dashed #0f3460;
|
||||
border-radius: 12px;
|
||||
padding: 60px 40px;
|
||||
text-align: center;
|
||||
margin-bottom: 40px;
|
||||
transition: all 0.3s;
|
||||
}
|
||||
.upload-zone:hover, .upload-zone.dragover {
|
||||
border-color: #e94560;
|
||||
background: rgba(233, 69, 96, 0.05);
|
||||
}
|
||||
.spinner {
|
||||
display: inline-block;
|
||||
width: 16px;
|
||||
height: 16px;
|
||||
border: 2px solid rgba(255,170,0,0.3);
|
||||
border-top-color: #ffaa00;
|
||||
border-radius: 50%;
|
||||
animation: spin 1s linear infinite;
|
||||
vertical-align: middle;
|
||||
margin-right: 6px;
|
||||
}
|
||||
@keyframes spin { to { transform: rotate(360deg); } }
|
||||
.processing-row { animation: pulse 2s infinite; }
|
||||
@keyframes pulse { 0%, 100% { opacity: 1; } 50% { opacity: 0.7; } }
|
||||
.delete-btn {
|
||||
background: transparent;
|
||||
border: none;
|
||||
color: #ff4444;
|
||||
cursor: pointer;
|
||||
font-size: 16px;
|
||||
margin-left: 10px;
|
||||
opacity: 0.5;
|
||||
}
|
||||
.delete-btn:hover { opacity: 1; }
|
||||
.upload-zone input { display: none; }
|
||||
.upload-btn {
|
||||
background: #e94560;
|
||||
color: white;
|
||||
border: none;
|
||||
padding: 12px 32px;
|
||||
border-radius: 6px;
|
||||
cursor: pointer;
|
||||
font-size: 16px;
|
||||
margin-top: 15px;
|
||||
}
|
||||
.projects-table {
|
||||
width: 100%;
|
||||
border-collapse: collapse;
|
||||
}
|
||||
.projects-table th {
|
||||
text-align: left;
|
||||
padding: 15px;
|
||||
border-bottom: 2px solid #0f3460;
|
||||
font-size: 13px;
|
||||
text-transform: uppercase;
|
||||
opacity: 0.7;
|
||||
}
|
||||
.projects-table td {
|
||||
padding: 15px;
|
||||
border-bottom: 1px solid #1a1a2e;
|
||||
}
|
||||
.projects-table tr:hover { background: rgba(255,255,255,0.03); }
|
||||
.status {
|
||||
display: inline-block;
|
||||
padding: 4px 12px;
|
||||
border-radius: 20px;
|
||||
font-size: 12px;
|
||||
font-weight: 600;
|
||||
}
|
||||
.status-completed { background: rgba(0,200,100,0.2); color: #00c864; }
|
||||
.status-processing { background: rgba(255,170,0,0.2); color: #ffaa00; }
|
||||
.status-error { background: rgba(255,0,0,0.2); color: #ff4444; }
|
||||
.status-uploaded { background: rgba(100,100,255,0.2); color: #8888ff; }
|
||||
.view-link { color: #44aaff; text-decoration: none; }
|
||||
.view-link:hover { text-decoration: underline; }
|
||||
#progress-bar {
|
||||
display: none;
|
||||
width: 100%;
|
||||
height: 4px;
|
||||
background: #1a1a2e;
|
||||
border-radius: 2px;
|
||||
margin-top: 15px;
|
||||
overflow: hidden;
|
||||
}
|
||||
#progress-fill {
|
||||
width: 0%;
|
||||
height: 100%;
|
||||
background: #e94560;
|
||||
transition: width 0.3s;
|
||||
}
|
||||
.error-msg { color: #ff4444; margin-top: 10px; font-size: 14px; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<h1>Blueprint QC Dashboard</h1>
|
||||
|
||||
<div class="stats" id="stats">
|
||||
<div class="stat-card">
|
||||
<div class="stat-value" id="stat-projects">-</div>
|
||||
<div class="stat-label">Проектов</div>
|
||||
</div>
|
||||
<div class="stat-card">
|
||||
<div class="stat-value" id="stat-issues">-</div>
|
||||
<div class="stat-label">Замечаний</div>
|
||||
</div>
|
||||
<div class="stat-card">
|
||||
<div class="stat-value" id="stat-feedback">-</div>
|
||||
<div class="stat-label">Размечено</div>
|
||||
</div>
|
||||
<div class="stat-card">
|
||||
<div class="stat-value" id="stat-accuracy">-</div>
|
||||
<div class="stat-label">Точность</div>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div class="upload-zone" id="uploadZone">
|
||||
<h3>Перетащите PDF сюда или кликните для выбора</h3>
|
||||
<p style="opacity:0.6; margin-top:10px;">Максимальный размер: 100 МБ</p>
|
||||
<input type="file" id="fileInput" accept=".pdf">
|
||||
<button class="upload-btn" onclick="document.getElementById('fileInput').click()">Выбрать файл</button>
|
||||
<div id="progress-bar"><div id="progress-fill"></div></div>
|
||||
<div id="upload-error" class="error-msg"></div>
|
||||
</div>
|
||||
|
||||
<h2 style="margin-bottom:20px;">Проекты</h2>
|
||||
<table class="projects-table" id="projectsTable">
|
||||
<thead>
|
||||
<tr>
|
||||
<th>ID</th>
|
||||
<th>Название</th>
|
||||
<th>Файл</th>
|
||||
<th>Статус</th>
|
||||
<th>Создан</th>
|
||||
<th>Замечаний</th>
|
||||
<th>Действия</th>
|
||||
</tr>
|
||||
</thead>
|
||||
<tbody></tbody>
|
||||
</table>
|
||||
</div>
|
||||
|
||||
<script>
|
||||
const API = '/api';
|
||||
|
||||
// Stats
|
||||
async function loadStats() {
|
||||
try {
|
||||
const res = await fetch(`${API}/stats`);
|
||||
const data = await res.json();
|
||||
document.getElementById('stat-projects').textContent = data.total_projects;
|
||||
document.getElementById('stat-issues').textContent = data.total_issues;
|
||||
document.getElementById('stat-feedback').textContent = data.feedback_stats.true_positive + data.feedback_stats.false_positive;
|
||||
document.getElementById('stat-accuracy').textContent = data.accuracy_estimate ? (data.accuracy_estimate * 100).toFixed(1) + '%' : '-';
|
||||
} catch (e) { console.error('Stats error:', e); }
|
||||
}
|
||||
|
||||
// Track previous statuses for detecting completion
|
||||
let prevStatuses = {};
|
||||
|
||||
// Projects
|
||||
async function loadProjects() {
|
||||
try {
|
||||
const res = await fetch(`${API}/projects`);
|
||||
const projects = await res.json();
|
||||
const tbody = document.querySelector('#projectsTable tbody');
|
||||
|
||||
// Check for newly completed projects
|
||||
projects.forEach(p => {
|
||||
if (prevStatuses[p.id] === 'processing' && p.status === 'completed') {
|
||||
showToast(`Project "${p.name || p.pdf_filename}" обработан!`, 'success');
|
||||
}
|
||||
prevStatuses[p.id] = p.status;
|
||||
});
|
||||
|
||||
tbody.innerHTML = projects.map(p => {
|
||||
const isProcessing = p.status === 'processing' || p.status === 'uploaded';
|
||||
const statusHtml = isProcessing
|
||||
? `<span class="status status-${p.status}"><span class="spinner"></span>${p.status}</span>`
|
||||
: `<span class="status status-${p.status}">${p.status}</span>`;
|
||||
let actions;
|
||||
if (p.status === 'completed') {
|
||||
actions = `<a href="/viewer/${p.id}/1" class="view-link" target="_blank">Viewer</a> | <a href="/review.html?project=${p.id}" class="view-link">Review</a> <button class="delete-btn" onclick="deleteProject(${p.id})" title="Удалить">🗑</button>`;
|
||||
} else if (p.status === 'processing') {
|
||||
actions = '<span style="opacity:0.5"><span class="spinner"></span> Обработка...</span>';
|
||||
} else {
|
||||
actions = `
|
||||
<label style="font-size:11px; margin-right:6px; cursor:pointer;">
|
||||
<input type="checkbox" id="tiling-${p.id}" style="vertical-align:middle;"> tiling
|
||||
</label>
|
||||
<button class="upload-btn" style="padding:4px 12px; font-size:12px;" onclick="analyzeProject(${p.id})">🔬 Анализировать</button>
|
||||
<button class="delete-btn" onclick="deleteProject(${p.id})" title="Удалить">🗑</button>
|
||||
`;
|
||||
}
|
||||
return `
|
||||
<tr class="${isProcessing ? 'processing-row' : ''}">
|
||||
<td>${p.id}</td>
|
||||
<td>${p.name || '-'}</td>
|
||||
<td>${p.pdf_filename}</td>
|
||||
<td>${statusHtml}</td>
|
||||
<td>${new Date(p.created_at).toLocaleString()}</td>
|
||||
<td>${p.issues ? p.issues.length : (isProcessing ? '<span class="spinner"></span>' : '0')}</td>
|
||||
<td>${actions}</td>
|
||||
</tr>
|
||||
`}).join('');
|
||||
} catch (e) { console.error('Projects error:', e); }
|
||||
}
|
||||
|
||||
async function deleteProject(id) {
|
||||
if (!confirm('Удалить проект?')) return;
|
||||
try {
|
||||
const res = await fetch(`${API}/projects/${id}`, { method: 'DELETE' });
|
||||
if (res.ok) {
|
||||
loadProjects();
|
||||
loadStats();
|
||||
showToast('Проект удалён', 'success');
|
||||
}
|
||||
} catch(e) { showToast('Ошибка удаления', 'error'); }
|
||||
}
|
||||
|
||||
// Upload
|
||||
const uploadZone = document.getElementById('uploadZone');
|
||||
const fileInput = document.getElementById('fileInput');
|
||||
const progressBar = document.getElementById('progress-bar');
|
||||
const progressFill = document.getElementById('progress-fill');
|
||||
const uploadError = document.getElementById('upload-error');
|
||||
|
||||
uploadZone.addEventListener('dragover', (e) => { e.preventDefault(); uploadZone.classList.add('dragover'); });
|
||||
uploadZone.addEventListener('dragleave', () => uploadZone.classList.remove('dragover'));
|
||||
uploadZone.addEventListener('drop', (e) => {
|
||||
e.preventDefault();
|
||||
uploadZone.classList.remove('dragover');
|
||||
if (e.dataTransfer.files.length) handleFile(e.dataTransfer.files[0]);
|
||||
});
|
||||
|
||||
fileInput.addEventListener('change', (e) => { if (e.target.files.length) handleFile(e.target.files[0]); });
|
||||
|
||||
async function analyzeProject(id) {
|
||||
try {
|
||||
const useTiling = document.getElementById(`tiling-${id}`)?.checked || false;
|
||||
const url = useTiling ? `${API}/projects/${id}/analyze?use_tiling=true` : `${API}/projects/${id}/analyze`;
|
||||
const res = await fetch(url, { method: 'POST' });
|
||||
if (res.ok) {
|
||||
const data = await res.json();
|
||||
showToast(`Анализ запущен (${data.ocr_engine})!`, 'success');
|
||||
loadProjects();
|
||||
} else {
|
||||
const err = await res.text();
|
||||
showToast('Ошибка запуска: ' + err, 'error');
|
||||
}
|
||||
} catch(e) { showToast('Ошибка сети', 'error'); }
|
||||
}
|
||||
|
||||
async function handleFile(file) {
|
||||
if (!file.name.endsWith('.pdf')) {
|
||||
uploadError.textContent = 'Только PDF файлы';
|
||||
return;
|
||||
}
|
||||
uploadError.textContent = '';
|
||||
progressBar.style.display = 'block';
|
||||
progressFill.style.width = '30%';
|
||||
|
||||
const form = new FormData();
|
||||
form.append('file', file);
|
||||
form.append('name', file.name);
|
||||
|
||||
try {
|
||||
const res = await fetch(`${API}/projects/upload`, { method: 'POST', body: form });
|
||||
progressFill.style.width = '100%';
|
||||
if (res.ok) {
|
||||
setTimeout(() => { progressBar.style.display = 'none'; progressFill.style.width = '0%'; loadProjects(); loadStats(); }, 500);
|
||||
} else {
|
||||
uploadError.textContent = 'Ошибка загрузки: ' + res.statusText;
|
||||
progressBar.style.display = 'none';
|
||||
}
|
||||
} catch (e) {
|
||||
uploadError.textContent = 'Ошибка сети';
|
||||
progressBar.style.display = 'none';
|
||||
}
|
||||
}
|
||||
|
||||
// Toast notification
|
||||
function showToast(msg, type) {
|
||||
const toast = document.createElement('div');
|
||||
toast.style.cssText = `position:fixed; bottom:20px; right:20px; background:rgba(0,0,0,0.95); color:white; padding:14px 24px; border-radius:8px; border:1px solid ${type==='success'?'#00c864':'#ff4444'}; z-index:100000; font-size:13px; box-shadow:0 4px 20px rgba(0,0,0,0.5);`;
|
||||
toast.textContent = msg;
|
||||
document.body.appendChild(toast);
|
||||
setTimeout(() => toast.remove(), 4000);
|
||||
}
|
||||
|
||||
// Init
|
||||
loadStats();
|
||||
loadProjects();
|
||||
setInterval(() => { loadProjects(); loadStats(); }, 5000); // Обновление каждые 5 сек
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
495
backend/static/review.html
Normal file
495
backend/static/review.html
Normal file
@ -0,0 +1,495 @@
|
||||
<!DOCTYPE html>
|
||||
<html lang="ru">
|
||||
<head>
|
||||
<meta charset="UTF-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1.0">
|
||||
<title>Review Issues — Blueprint QC</title>
|
||||
<style>
|
||||
* { margin: 0; padding: 0; box-sizing: border-box; }
|
||||
body {
|
||||
font-family: 'Segoe UI', system-ui, sans-serif;
|
||||
background: #0d0d1a;
|
||||
color: #eee;
|
||||
min-height: 100vh;
|
||||
}
|
||||
.container { max-width: 1400px; margin: 0 auto; padding: 20px; }
|
||||
header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 30px;
|
||||
padding-bottom: 20px;
|
||||
border-bottom: 2px solid #0f3460;
|
||||
}
|
||||
h1 { color: #e94560; }
|
||||
.project-info { opacity: 0.7; font-size: 14px; }
|
||||
|
||||
.filters {
|
||||
display: flex;
|
||||
gap: 15px;
|
||||
margin-bottom: 25px;
|
||||
flex-wrap: wrap;
|
||||
align-items: center;
|
||||
}
|
||||
.filter-group {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
align-items: center;
|
||||
}
|
||||
.filter-group label { font-size: 13px; opacity: 0.8; }
|
||||
.filter-group select, .filter-group input {
|
||||
background: #1a1a2e;
|
||||
border: 1px solid #0f3460;
|
||||
color: #eee;
|
||||
padding: 6px 12px;
|
||||
border-radius: 4px;
|
||||
font-size: 13px;
|
||||
}
|
||||
.filter-btn {
|
||||
background: #e94560;
|
||||
border: none;
|
||||
color: white;
|
||||
padding: 6px 16px;
|
||||
border-radius: 4px;
|
||||
cursor: pointer;
|
||||
font-size: 13px;
|
||||
}
|
||||
.filter-btn.secondary {
|
||||
background: #1a1a2e;
|
||||
border: 1px solid #0f3460;
|
||||
}
|
||||
|
||||
.progress-bar {
|
||||
width: 100%;
|
||||
height: 6px;
|
||||
background: #1a1a2e;
|
||||
border-radius: 3px;
|
||||
margin-bottom: 25px;
|
||||
overflow: hidden;
|
||||
}
|
||||
.progress-fill {
|
||||
height: 100%;
|
||||
background: linear-gradient(90deg, #e94560, #ffaa00);
|
||||
transition: width 0.5s;
|
||||
}
|
||||
.progress-text {
|
||||
font-size: 12px;
|
||||
opacity: 0.6;
|
||||
margin-bottom: 8px;
|
||||
}
|
||||
|
||||
.issues-grid {
|
||||
display: grid;
|
||||
grid-template-columns: repeat(auto-fill, minmax(350px, 1fr));
|
||||
gap: 15px;
|
||||
}
|
||||
.issue-card {
|
||||
background: #16213e;
|
||||
border-left: 4px solid var(--card-color, #888);
|
||||
border-radius: 8px;
|
||||
padding: 15px;
|
||||
transition: all 0.2s;
|
||||
position: relative;
|
||||
}
|
||||
.issue-card:hover {
|
||||
background: #1a2847;
|
||||
transform: translateY(-2px);
|
||||
}
|
||||
.issue-card.reviewed-tp { border-left-color: #00c864; }
|
||||
.issue-card.reviewed-fp { opacity: 0.6; }
|
||||
.issue-card.reviewed-fp .issue-text { text-decoration: line-through; }
|
||||
.issue-card.reviewed-ns { border-left-color: #888; }
|
||||
|
||||
.issue-header {
|
||||
display: flex;
|
||||
justify-content: space-between;
|
||||
align-items: center;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
.issue-id {
|
||||
font-size: 11px;
|
||||
background: rgba(255,255,255,0.1);
|
||||
padding: 2px 8px;
|
||||
border-radius: 10px;
|
||||
}
|
||||
.issue-type {
|
||||
font-size: 11px;
|
||||
text-transform: uppercase;
|
||||
opacity: 0.7;
|
||||
}
|
||||
.issue-severity {
|
||||
font-size: 10px;
|
||||
padding: 2px 8px;
|
||||
border-radius: 10px;
|
||||
font-weight: bold;
|
||||
}
|
||||
.severity-error { background: rgba(255,0,0,0.2); color: #ff4444; }
|
||||
.severity-warning { background: rgba(255,170,0,0.2); color: #ffaa00; }
|
||||
.severity-info { background: rgba(0,150,255,0.2); color: #44aaff; }
|
||||
|
||||
.issue-text {
|
||||
font-size: 13px;
|
||||
line-height: 1.5;
|
||||
margin-bottom: 10px;
|
||||
}
|
||||
.issue-meta {
|
||||
font-size: 11px;
|
||||
opacity: 0.6;
|
||||
margin-bottom: 12px;
|
||||
}
|
||||
.issue-meta span { margin-right: 15px; }
|
||||
|
||||
.feedback-bar {
|
||||
display: flex;
|
||||
gap: 8px;
|
||||
}
|
||||
.feedback-bar button {
|
||||
flex: 1;
|
||||
padding: 8px;
|
||||
border: 1px solid #333;
|
||||
background: #0d0d1a;
|
||||
color: #ccc;
|
||||
border-radius: 6px;
|
||||
cursor: pointer;
|
||||
font-size: 12px;
|
||||
transition: all 0.2s;
|
||||
}
|
||||
.feedback-bar button:hover {
|
||||
background: #1a1a2e;
|
||||
color: #fff;
|
||||
}
|
||||
.feedback-bar button.tp:hover { background: rgba(0,200,100,0.2); border-color: #00c864; color: #00c864; }
|
||||
.feedback-bar button.fp:hover { background: rgba(255,0,0,0.2); border-color: #ff4444; color: #ff4444; }
|
||||
.feedback-bar button.ns:hover { background: rgba(150,150,150,0.2); border-color: #888; color: #888; }
|
||||
.feedback-bar button:disabled { opacity: 0.5; cursor: not-allowed; }
|
||||
|
||||
.reviewed-badge {
|
||||
position: absolute;
|
||||
top: 10px;
|
||||
right: 10px;
|
||||
font-size: 11px;
|
||||
padding: 2px 8px;
|
||||
border-radius: 10px;
|
||||
font-weight: bold;
|
||||
}
|
||||
.badge-tp { background: rgba(0,200,100,0.2); color: #00c864; }
|
||||
.badge-fp { background: rgba(255,0,0,0.2); color: #ff4444; }
|
||||
.badge-ns { background: rgba(150,150,150,0.2); color: #888; }
|
||||
|
||||
.empty-state {
|
||||
text-align: center;
|
||||
padding: 80px 20px;
|
||||
opacity: 0.5;
|
||||
}
|
||||
.empty-state h2 { margin-bottom: 15px; }
|
||||
|
||||
.viewer-link {
|
||||
color: #44aaff;
|
||||
text-decoration: none;
|
||||
font-size: 12px;
|
||||
}
|
||||
.viewer-link:hover { text-decoration: underline; }
|
||||
|
||||
#toast {
|
||||
position: fixed;
|
||||
bottom: 20px;
|
||||
right: 20px;
|
||||
background: rgba(0,0,0,0.95);
|
||||
color: white;
|
||||
padding: 14px 24px;
|
||||
border-radius: 8px;
|
||||
border: 1px solid #e94560;
|
||||
z-index: 100000;
|
||||
display: none;
|
||||
font-size: 13px;
|
||||
box-shadow: 0 4px 20px rgba(0,0,0,0.5);
|
||||
}
|
||||
|
||||
.stats-row {
|
||||
display: flex;
|
||||
gap: 20px;
|
||||
margin-bottom: 20px;
|
||||
flex-wrap: wrap;
|
||||
}
|
||||
.stat-pill {
|
||||
background: #1a1a2e;
|
||||
padding: 8px 16px;
|
||||
border-radius: 20px;
|
||||
font-size: 13px;
|
||||
}
|
||||
.stat-pill .num { font-weight: bold; color: #e94560; }
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<div class="container">
|
||||
<header>
|
||||
<div>
|
||||
<h1>Review Issues</h1>
|
||||
<div class="project-info" id="projectInfo">Loading...</div>
|
||||
</div>
|
||||
<a href="/" style="color:#44aaff; text-decoration:none;">← Dashboard</a>
|
||||
</header>
|
||||
|
||||
<div class="stats-row" id="statsRow"></div>
|
||||
|
||||
<div class="progress-text" id="progressText">0% reviewed</div>
|
||||
<div class="progress-bar">
|
||||
<div class="progress-fill" id="progressFill" style="width:0%"></div>
|
||||
</div>
|
||||
|
||||
<div class="filters">
|
||||
<div class="filter-group">
|
||||
<label>Project:</label>
|
||||
<select id="projectSelect"><option value="">Select...</option></select>
|
||||
</div>
|
||||
<div class="filter-group">
|
||||
<label>Severity:</label>
|
||||
<select id="severityFilter">
|
||||
<option value="all">All</option>
|
||||
<option value="error">Error</option>
|
||||
<option value="warning">Warning</option>
|
||||
<option value="info">Info</option>
|
||||
</select>
|
||||
</div>
|
||||
<div class="filter-group">
|
||||
<label>Type:</label>
|
||||
<select id="typeFilter"><option value="all">All</option></select>
|
||||
</div>
|
||||
<div class="filter-group">
|
||||
<label>Status:</label>
|
||||
<select id="statusFilter">
|
||||
<option value="all">All</option>
|
||||
<option value="unreviewed">Unreviewed</option>
|
||||
<option value="reviewed">Reviewed</option>
|
||||
</select>
|
||||
</div>
|
||||
<button class="filter-btn" onclick="loadIssues()">Apply</button>
|
||||
<button class="filter-btn secondary" onclick="exportTrainingData()">📥 Export Training Data</button>
|
||||
</div>
|
||||
|
||||
<div class="issues-grid" id="issuesGrid"></div>
|
||||
|
||||
<div class="empty-state" id="emptyState" style="display:none;">
|
||||
<h2>No issues found</h2>
|
||||
<p>Try adjusting filters or upload a new PDF.</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<div id="toast"></div>
|
||||
|
||||
<script>
|
||||
const API = '/api';
|
||||
let currentProjectId = null;
|
||||
let allIssues = [];
|
||||
let allProjects = [];
|
||||
|
||||
// Init
|
||||
async function init() {
|
||||
await loadProjects();
|
||||
// Check URL param
|
||||
const urlParams = new URLSearchParams(window.location.search);
|
||||
const pid = urlParams.get('project');
|
||||
if (pid) {
|
||||
document.getElementById('projectSelect').value = pid;
|
||||
await selectProject(pid);
|
||||
}
|
||||
}
|
||||
|
||||
async function loadProjects() {
|
||||
try {
|
||||
const res = await fetch(`${API}/projects`);
|
||||
allProjects = await res.json();
|
||||
const select = document.getElementById('projectSelect');
|
||||
select.innerHTML = '<option value="">Select project...</option>' +
|
||||
allProjects.map(p => `<option value="${p.id}">${p.name || p.pdf_filename} (#${p.id})</option>`).join('');
|
||||
select.addEventListener('change', (e) => selectProject(e.target.value));
|
||||
} catch(e) { console.error(e); }
|
||||
}
|
||||
|
||||
async function selectProject(pid) {
|
||||
if (!pid) return;
|
||||
currentProjectId = pid;
|
||||
const project = allProjects.find(p => p.id == pid);
|
||||
if (project) {
|
||||
document.getElementById('projectInfo').textContent =
|
||||
`${project.name || project.pdf_filename} | Status: ${project.status}`;
|
||||
}
|
||||
await loadIssues();
|
||||
}
|
||||
|
||||
async function loadIssues() {
|
||||
if (!currentProjectId) {
|
||||
showToast('Select a project first', 'warning');
|
||||
return;
|
||||
}
|
||||
|
||||
const severity = document.getElementById('severityFilter').value;
|
||||
const typeFilter = document.getElementById('typeFilter').value;
|
||||
const statusFilter = document.getElementById('statusFilter').value;
|
||||
|
||||
try {
|
||||
let url = `${API}/projects/${currentProjectId}/issues`;
|
||||
const params = [];
|
||||
if (severity !== 'all') params.push(`severity=${severity}`);
|
||||
if (typeFilter !== 'all') params.push(`issue_type=${typeFilter}`);
|
||||
if (statusFilter === 'reviewed') params.push(`has_feedback=true`);
|
||||
if (statusFilter === 'unreviewed') params.push(`has_feedback=false`);
|
||||
if (params.length) url += '?' + params.join('&');
|
||||
|
||||
const res = await fetch(url);
|
||||
allIssues = await res.json();
|
||||
renderIssues();
|
||||
updateStats();
|
||||
updateTypeFilter();
|
||||
} catch(e) { showToast('Error loading issues: ' + e.message, 'error'); }
|
||||
}
|
||||
|
||||
function renderIssues() {
|
||||
const grid = document.getElementById('issuesGrid');
|
||||
const empty = document.getElementById('emptyState');
|
||||
|
||||
if (!allIssues.length) {
|
||||
grid.innerHTML = '';
|
||||
empty.style.display = 'block';
|
||||
return;
|
||||
}
|
||||
empty.style.display = 'none';
|
||||
|
||||
grid.innerHTML = allIssues.map(issue => {
|
||||
const color = issue.severity === 'error' ? '#ff4444' : (issue.severity === 'warning' ? '#ffaa00' : '#44aaff');
|
||||
const reviewedClass = issue.feedback ?
|
||||
(issue.feedback.is_true_positive === true ? 'reviewed-tp' :
|
||||
(issue.feedback.is_true_positive === false ? 'reviewed-fp' : 'reviewed-ns')) : '';
|
||||
const badge = issue.feedback ?
|
||||
(issue.feedback.is_true_positive === true ? '<span class="reviewed-badge badge-tp">✅ TP</span>' :
|
||||
(issue.feedback.is_true_positive === false ? '<span class="reviewed-badge badge-fp">❌ FP</span>' :
|
||||
'<span class="reviewed-badge badge-ns">🤷 NS</span>')) : '';
|
||||
|
||||
return `
|
||||
<div class="issue-card ${reviewedClass}" style="--card-color:${color}" data-issue-id="${issue.id}">
|
||||
${badge}
|
||||
<div class="issue-header">
|
||||
<span class="issue-id">#${issue.id}</span>
|
||||
<span class="issue-severity severity-${issue.severity}">${issue.severity}</span>
|
||||
</div>
|
||||
<div class="issue-type">${issue.issue_type}</div>
|
||||
<div class="issue-text">${issue.message}</div>
|
||||
<div class="issue-meta">
|
||||
<span>Page ${issue.page_number || '?'}</span>
|
||||
<span>Conf: ${issue.confidence ? issue.confidence.toFixed(2) : 'N/A'}</span>
|
||||
${issue.dimension_text ? `<span>Dim: "${issue.dimension_text}"</span>` : ''}
|
||||
<a href="/viewer/${issue.project_id}/${issue.page_number || 1}" class="viewer-link" target="_blank">Open Viewer →</a>
|
||||
</div>
|
||||
<div class="feedback-bar">
|
||||
<button class="tp" onclick="submitFeedback(${issue.id}, true, this)" ${issue.feedback ? 'disabled' : ''}>✅ Real Issue</button>
|
||||
<button class="fp" onclick="submitFeedback(${issue.id}, false, this)" ${issue.feedback ? 'disabled' : ''}>❌ False Positive</button>
|
||||
<button class="ns" onclick="submitFeedback(${issue.id}, null, this)" ${issue.feedback ? 'disabled' : ''}>🤷 Not Sure</button>
|
||||
</div>
|
||||
</div>
|
||||
`;
|
||||
}).join('');
|
||||
}
|
||||
|
||||
async function submitFeedback(issueId, isTP, btn) {
|
||||
const card = btn.closest('.issue-card');
|
||||
const btns = card.querySelectorAll('.feedback-bar button');
|
||||
btns.forEach(b => b.disabled = true);
|
||||
btn.textContent = '...';
|
||||
|
||||
try {
|
||||
const res = await fetch(`${API}/feedback`, {
|
||||
method: 'POST',
|
||||
headers: {'Content-Type': 'application/json'},
|
||||
body: JSON.stringify({
|
||||
issue_id: issueId,
|
||||
is_true_positive: isTP,
|
||||
action_taken: isTP === true ? 'fixed' : (isTP === false ? 'ignored' : 'not_sure')
|
||||
})
|
||||
});
|
||||
|
||||
if (res.ok) {
|
||||
// Update the issue in memory
|
||||
const issue = allIssues.find(i => i.id === issueId);
|
||||
if (issue) {
|
||||
issue.feedback = { is_true_positive: isTP };
|
||||
}
|
||||
renderIssues();
|
||||
updateStats();
|
||||
showToast('Saved!', 'success');
|
||||
} else {
|
||||
const err = await res.text();
|
||||
showToast('Error: ' + err, 'error');
|
||||
btns.forEach(b => b.disabled = false);
|
||||
btn.textContent = isTP === true ? '✅ Real Issue' : (isTP === false ? '❌ False Positive' : '🤷 Not Sure');
|
||||
}
|
||||
} catch(e) {
|
||||
showToast('Network error: ' + e.message, 'error');
|
||||
btns.forEach(b => b.disabled = false);
|
||||
btn.textContent = isTP === true ? '✅ Real Issue' : (isTP === false ? '❌ False Positive' : '🤷 Not Sure');
|
||||
}
|
||||
}
|
||||
|
||||
function updateStats() {
|
||||
const total = allIssues.length;
|
||||
const reviewed = allIssues.filter(i => i.feedback).length;
|
||||
const tp = allIssues.filter(i => i.feedback && i.feedback.is_true_positive === true).length;
|
||||
const fp = allIssues.filter(i => i.feedback && i.feedback.is_true_positive === false).length;
|
||||
const pct = total ? Math.round((reviewed / total) * 100) : 0;
|
||||
|
||||
document.getElementById('progressFill').style.width = pct + '%';
|
||||
document.getElementById('progressText').textContent =
|
||||
`${pct}% reviewed (${reviewed}/${total}) | ✅ TP: ${tp} | ❌ FP: ${fp}`;
|
||||
|
||||
document.getElementById('statsRow').innerHTML = `
|
||||
<div class="stat-pill">Total: <span class="num">${total}</span></div>
|
||||
<div class="stat-pill">Reviewed: <span class="num">${reviewed}</span></div>
|
||||
<div class="stat-pill">True Positive: <span class="num" style="color:#00c864">${tp}</span></div>
|
||||
<div class="stat-pill">False Positive: <span class="num" style="color:#ff4444">${fp}</span></div>
|
||||
`;
|
||||
}
|
||||
|
||||
function updateTypeFilter() {
|
||||
const types = [...new Set(allIssues.map(i => i.issue_type))];
|
||||
const select = document.getElementById('typeFilter');
|
||||
const current = select.value;
|
||||
select.innerHTML = '<option value="all">All</option>' +
|
||||
types.map(t => `<option value="${t}">${t}</option>`).join('');
|
||||
if (types.includes(current)) select.value = current;
|
||||
}
|
||||
|
||||
async function exportTrainingData() {
|
||||
if (!currentProjectId) {
|
||||
showToast('Select a project first', 'warning');
|
||||
return;
|
||||
}
|
||||
try {
|
||||
const res = await fetch(`${API}/training/export?project_id=${currentProjectId}&only_labeled=true`);
|
||||
const data = await res.json();
|
||||
|
||||
// Download as file
|
||||
const blob = new Blob([JSON.stringify(data, null, 2)], {type: 'application/json'});
|
||||
const url = URL.createObjectURL(blob);
|
||||
const a = document.createElement('a');
|
||||
a.href = url;
|
||||
a.download = `training_data_project_${currentProjectId}_${new Date().toISOString().slice(0,10)}.json`;
|
||||
a.click();
|
||||
URL.revokeObjectURL(url);
|
||||
|
||||
showToast(`Exported ${data.total_samples} samples!`, 'success');
|
||||
} catch(e) {
|
||||
showToast('Export error: ' + e.message, 'error');
|
||||
}
|
||||
}
|
||||
|
||||
function showToast(msg, type) {
|
||||
const toast = document.getElementById('toast');
|
||||
toast.textContent = msg;
|
||||
toast.style.borderColor = type === 'success' ? '#00c864' : (type === 'error' ? '#ff4444' : '#ffaa00');
|
||||
toast.style.display = 'block';
|
||||
setTimeout(() => toast.style.display = 'none', 3000);
|
||||
}
|
||||
|
||||
init();
|
||||
</script>
|
||||
</body>
|
||||
</html>
|
||||
Loading…
Reference in New Issue
Block a user