Compare commits
12 Commits
93668273ff
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
| 9780311cd0 | |||
| f2bb88f040 | |||
| 985768f400 | |||
| 2761282c0b | |||
| b7269601eb | |||
| b2a7c1a2e3 | |||
| 6274b4a0b8 | |||
| 351dd3b608 | |||
| 8a0edc3d59 | |||
| 71513ea62c | |||
| 7ef93276e1 | |||
| eb1fb5ca78 |
@ -10,7 +10,23 @@
|
||||
"Bash(/home/anthony/go/bin/go build *)",
|
||||
"Bash(npm install *)",
|
||||
"Bash(/home/anthony/.config/JetBrains/WebStorm2026.1/node/versions/24.14.1/bin/npm install *)",
|
||||
"Bash(npm run *)"
|
||||
"Bash(npm run *)",
|
||||
"Bash(go build *)",
|
||||
"Bash(npx tsc *)",
|
||||
"Bash(node_modules/.bin/tsc --noEmit)",
|
||||
"Bash(/usr/bin/node node_modules/.bin/tsc --noEmit)",
|
||||
"Bash(fish -c \"which node\")",
|
||||
"Read(//opt/**)",
|
||||
"Bash(/home/anthony/.config/JetBrains/WebStorm2026.1/node/versions/24.14.1/bin/node node_modules/.bin/tsc --noEmit)",
|
||||
"Bash(chmod +x /home/anthony/Documents/Projects/Tradarr/build-push.sh)",
|
||||
"Bash(fish -c \"npm install react-markdown\")",
|
||||
"Bash(fish -c \"which npm; which pnpm; which bun\")",
|
||||
"Bash(/usr/bin/npm run *)",
|
||||
"Bash(/home/anthony/.config/JetBrains/WebStorm2026.1/node/versions/24.14.1/lib/node_modules/npm/bin/npm run *)",
|
||||
"WebFetch(domain:docs.ollama.com)",
|
||||
"WebFetch(domain:github.com)",
|
||||
"Bash(docker compose *)",
|
||||
"Bash(sudo docker *)"
|
||||
]
|
||||
}
|
||||
}
|
||||
|
||||
3
.gitignore
vendored
3
.gitignore
vendored
@ -28,3 +28,6 @@ Thumbs.db
|
||||
|
||||
# Logs
|
||||
*.log
|
||||
|
||||
# Scripts de déploiement
|
||||
build-push.sh
|
||||
|
||||
154
README.md
154
README.md
@ -0,0 +1,154 @@
|
||||
# Tradarr
|
||||
|
||||
Agrégateur d'actualités financières avec résumés IA personnalisés par utilisateur.
|
||||
|
||||
Tradarr scrape des sources d'information (Bloomberg, StockTwits, Reuters, WatcherGuru), croise les articles avec la watchlist de chaque trader, puis génère via IA un résumé structuré orienté trading. L'utilisateur peut ensuite sélectionner des extraits du résumé et poser des questions à l'IA pour approfondir une analyse.
|
||||
|
||||
## Fonctionnalités
|
||||
|
||||
- Scraping automatique de plusieurs sources financières (planifiable)
|
||||
- Résumés IA personnalisés selon la watchlist de chaque utilisateur
|
||||
- Filtrage intelligent des articles par pertinence (passe 1) avant résumé (passe 2)
|
||||
- Questions/réponses IA sur des extraits de résumés (rapports)
|
||||
- Support de plusieurs fournisseurs IA : OpenAI, Anthropic, Gemini, Ollama, Claude Code CLI
|
||||
- Interface PWA (installable sur mobile)
|
||||
- Panel d'administration complet
|
||||
|
||||
## Installation (production)
|
||||
|
||||
### Prérequis
|
||||
|
||||
- Docker et Docker Compose installés sur le serveur
|
||||
- Si tu utilises **Claude Code** comme fournisseur IA : Claude Code CLI installé et authentifié sur la machine hôte (`claude login`)
|
||||
|
||||
### 1. Créer le fichier `.env`
|
||||
|
||||
```bash
|
||||
cp .env.example .env
|
||||
```
|
||||
|
||||
Édite `.env` avec tes valeurs :
|
||||
|
||||
```env
|
||||
# PostgreSQL
|
||||
POSTGRES_DB=tradarr
|
||||
POSTGRES_USER=tradarr
|
||||
POSTGRES_PASSWORD=<mot_de_passe_fort>
|
||||
|
||||
# Secret JWT pour les tokens de session
|
||||
JWT_SECRET=<chaîne_aléatoire_longue>
|
||||
|
||||
# Clé de chiffrement AES-256 pour les clés API et credentials (32 bytes en hex)
|
||||
# Générer avec : openssl rand -hex 32
|
||||
ENCRYPTION_KEY=<32_bytes_en_hex>
|
||||
|
||||
# Compte administrateur créé automatiquement au premier démarrage
|
||||
ADMIN_EMAIL=admin@example.com
|
||||
ADMIN_PASSWORD=<mot_de_passe_fort>
|
||||
|
||||
# Port exposé sur le serveur (défaut : 80)
|
||||
FRONTEND_PORT=80
|
||||
```
|
||||
|
||||
### 2. Ajuster `docker-compose.prod.yml` si nécessaire
|
||||
|
||||
#### Fournisseur Claude Code CLI (optionnel)
|
||||
|
||||
Si tu veux utiliser **Claude Code** comme fournisseur IA, le backend doit accéder aux credentials de ta session Claude. La configuration par défaut monte `/home/anthony/.claude` — adapte ce chemin à l'utilisateur qui exécute Docker sur ton serveur :
|
||||
|
||||
```yaml
|
||||
# dans docker-compose.prod.yml, section backend > volumes
|
||||
volumes:
|
||||
- /home/<ton_user>/.claude:/root/.claude
|
||||
```
|
||||
|
||||
#### Images Docker
|
||||
|
||||
Les images sont référencées par tag dans `docker-compose.prod.yml`. Mets à jour les tags selon la version à déployer :
|
||||
|
||||
```yaml
|
||||
backend:
|
||||
image: gitea.anthonybouteiller.ovh/blomios/tradarr-backend:v1.0.0
|
||||
frontend:
|
||||
image: gitea.anthonybouteiller.ovh/blomios/tradarr-frontend:v1.0.0
|
||||
scraper:
|
||||
image: gitea.anthonybouteiller.ovh/blomios/tradarr-scraper:v1.0.0
|
||||
```
|
||||
|
||||
### 3. Démarrer
|
||||
|
||||
```bash
|
||||
docker compose -f docker-compose.prod.yml up -d
|
||||
```
|
||||
|
||||
L'interface est accessible sur `http://<serveur>:<FRONTEND_PORT>`.
|
||||
|
||||
---
|
||||
|
||||
## Configuration post-démarrage (panel admin)
|
||||
|
||||
Connecte-toi avec le compte admin défini dans `.env`, puis accède à **Admin** pour configurer :
|
||||
|
||||
### Fournisseurs IA
|
||||
|
||||
Ajoute au moins un fournisseur IA et assigne-le aux trois rôles disponibles :
|
||||
|
||||
| Rôle | Description |
|
||||
|------|-------------|
|
||||
| Résumés | Génération du résumé quotidien (passe 2) |
|
||||
| Rapports | Réponses aux questions sur les résumés |
|
||||
| Filtre articles | Sélection des articles pertinents (passe 1) |
|
||||
|
||||
Fournisseurs supportés :
|
||||
|
||||
| Nom | Clé API requise | Endpoint |
|
||||
|-----|----------------|----------|
|
||||
| `openai` | Oui | — |
|
||||
| `anthropic` | Oui | — |
|
||||
| `gemini` | Oui | — |
|
||||
| `ollama` | Non | `http://ollama:11434` (par défaut) |
|
||||
| `claudecode` | Non | — (utilise le CLI local) |
|
||||
|
||||
### Paramètres
|
||||
|
||||
| Clé | Description | Défaut |
|
||||
|-----|-------------|--------|
|
||||
| `articles_lookback_hours` | Fenêtre de temps pour récupérer les articles | `24` |
|
||||
| `summary_max_articles` | Nombre max d'articles envoyés au modèle pour le résumé | `50` |
|
||||
| `filter_batch_size` | Taille des lots pour la passe de filtrage IA | `20` |
|
||||
| `timezone` | Fuseau horaire affiché dans les résumés | `UTC` |
|
||||
| `ai_system_prompt` | Prompt système pour la génération des résumés | prompt par défaut |
|
||||
|
||||
### Planning
|
||||
|
||||
Configure les créneaux de génération automatique des résumés (jours et heures).
|
||||
|
||||
### Sources
|
||||
|
||||
Active ou désactive les sources de scraping :
|
||||
- **Bloomberg** — nécessite des credentials (onglet Credentials)
|
||||
- **StockTwits** — public, aucune configuration requise
|
||||
- **Reuters** — public
|
||||
- **WatcherGuru** — public
|
||||
|
||||
### Credentials Bloomberg
|
||||
|
||||
Renseigne ton login/mot de passe Bloomberg pour activer le scraping de cette source.
|
||||
|
||||
---
|
||||
|
||||
## Développement local
|
||||
|
||||
```bash
|
||||
# Démarrer tous les services
|
||||
docker compose up -d
|
||||
|
||||
# Rebuild après modification du code
|
||||
docker compose build backend frontend && docker compose up -d backend frontend
|
||||
```
|
||||
|
||||
Logs :
|
||||
```bash
|
||||
docker compose logs -f backend
|
||||
docker compose logs -f frontend
|
||||
```
|
||||
|
||||
@ -9,15 +9,18 @@ RUN CGO_ENABLED=0 GOOS=linux go build -o tradarr ./cmd/server
|
||||
|
||||
FROM debian:bookworm-slim
|
||||
|
||||
# Chromium pour le scraping Bloomberg
|
||||
# Chromium pour le scraping Bloomberg + Node.js pour Claude Code CLI
|
||||
RUN apt-get update && apt-get install -y \
|
||||
chromium \
|
||||
chromium-driver \
|
||||
ca-certificates \
|
||||
fonts-liberation \
|
||||
libnss3 \
|
||||
nodejs \
|
||||
npm \
|
||||
--no-install-recommends && \
|
||||
rm -rf /var/lib/apt/lists/*
|
||||
rm -rf /var/lib/apt/lists/* && \
|
||||
npm install -g @anthropic-ai/claude-code
|
||||
|
||||
WORKDIR /app
|
||||
COPY --from=builder /app/tradarr .
|
||||
|
||||
@ -15,6 +15,8 @@ import (
|
||||
"github.com/tradarr/backend/internal/scheduler"
|
||||
"github.com/tradarr/backend/internal/scraper"
|
||||
"github.com/tradarr/backend/internal/scraper/bloomberg"
|
||||
"github.com/tradarr/backend/internal/scraper/reuters"
|
||||
"github.com/tradarr/backend/internal/scraper/watcherguru"
|
||||
"github.com/tradarr/backend/internal/scraper/yahoofinance"
|
||||
)
|
||||
|
||||
@ -38,30 +40,23 @@ func main() {
|
||||
enc := crypto.New(cfg.EncryptionKey)
|
||||
pipeline := ai.NewPipeline(repo, enc)
|
||||
|
||||
// Créer le compte admin initial si nécessaire
|
||||
if err := ensureAdmin(repo, cfg); err != nil {
|
||||
log.Printf("ensure admin: %v", err)
|
||||
}
|
||||
|
||||
// Configurer les scrapers
|
||||
registry := scraper.NewRegistry(repo)
|
||||
registry.Register(bloomberg.NewDynamic(repo, enc, cfg.ScraperURL))
|
||||
registry.Register(yahoofinance.New())
|
||||
registry.Register(reuters.New())
|
||||
registry.Register(watcherguru.New())
|
||||
|
||||
// Bloomberg (credentials chargés depuis la DB à chaque run)
|
||||
bbScraper := bloomberg.NewDynamic(repo, enc, cfg.ChromePath)
|
||||
registry.Register(bbScraper)
|
||||
|
||||
stScraper := yahoofinance.New()
|
||||
registry.Register(stScraper)
|
||||
|
||||
// Scheduler
|
||||
sched := scheduler.New(registry, pipeline, repo)
|
||||
if err := sched.Start(); err != nil {
|
||||
log.Printf("scheduler: %v", err)
|
||||
}
|
||||
defer sched.Stop()
|
||||
|
||||
// API
|
||||
h := handlers.New(repo, cfg, enc, registry, pipeline)
|
||||
h := handlers.New(repo, cfg, enc, registry, pipeline, sched)
|
||||
r := api.SetupRouter(h, cfg.JWTSecret)
|
||||
|
||||
addr := fmt.Sprintf(":%s", cfg.Port)
|
||||
|
||||
@ -28,7 +28,7 @@ func newAnthropic(apiKey, model string) *anthropicProvider {
|
||||
|
||||
func (p *anthropicProvider) Name() string { return "anthropic" }
|
||||
|
||||
func (p *anthropicProvider) Summarize(ctx context.Context, prompt string) (string, error) {
|
||||
func (p *anthropicProvider) Summarize(ctx context.Context, prompt string, _ GenOptions) (string, error) {
|
||||
body := map[string]interface{}{
|
||||
"model": p.model,
|
||||
"max_tokens": 4096,
|
||||
|
||||
52
backend/internal/ai/claudecode.go
Normal file
52
backend/internal/ai/claudecode.go
Normal file
@ -0,0 +1,52 @@
|
||||
package ai
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"fmt"
|
||||
"os/exec"
|
||||
"strings"
|
||||
)
|
||||
|
||||
type claudeCodeProvider struct {
|
||||
model string
|
||||
}
|
||||
|
||||
func newClaudeCode(model string) *claudeCodeProvider {
|
||||
if model == "" {
|
||||
model = "claude-sonnet-4-6"
|
||||
}
|
||||
return &claudeCodeProvider{model: model}
|
||||
}
|
||||
|
||||
func (p *claudeCodeProvider) Name() string { return "claudecode" }
|
||||
|
||||
func (p *claudeCodeProvider) Summarize(ctx context.Context, prompt string, _ GenOptions) (string, error) {
|
||||
var stdout, stderr bytes.Buffer
|
||||
cmd := exec.CommandContext(ctx, "claude", "-p", prompt, "--model", p.model)
|
||||
cmd.Stdout = &stdout
|
||||
cmd.Stderr = &stderr
|
||||
|
||||
err := cmd.Run()
|
||||
fmt.Printf("[claudecode] stdout len=%d stderr=%q err=%v\n", stdout.Len(), stderr.String(), err)
|
||||
|
||||
if err != nil {
|
||||
msg := strings.TrimSpace(stderr.String())
|
||||
if msg == "" {
|
||||
msg = strings.TrimSpace(stdout.String())
|
||||
}
|
||||
if msg == "" {
|
||||
msg = err.Error()
|
||||
}
|
||||
return "", fmt.Errorf("claude cli: %s", msg)
|
||||
}
|
||||
return strings.TrimSpace(stdout.String()), nil
|
||||
}
|
||||
|
||||
func (p *claudeCodeProvider) ListModels(_ context.Context) ([]string, error) {
|
||||
return []string{
|
||||
"claude-opus-4-7",
|
||||
"claude-sonnet-4-6",
|
||||
"claude-haiku-4-5-20251001",
|
||||
}, nil
|
||||
}
|
||||
@ -28,7 +28,7 @@ func newGemini(apiKey, model string) *geminiProvider {
|
||||
|
||||
func (p *geminiProvider) Name() string { return "gemini" }
|
||||
|
||||
func (p *geminiProvider) Summarize(ctx context.Context, prompt string) (string, error) {
|
||||
func (p *geminiProvider) Summarize(ctx context.Context, prompt string, _ GenOptions) (string, error) {
|
||||
url := fmt.Sprintf(
|
||||
"https://generativelanguage.googleapis.com/v1beta/models/%s:generateContent?key=%s",
|
||||
p.model, p.apiKey,
|
||||
@ -74,11 +74,43 @@ func (p *geminiProvider) Summarize(ctx context.Context, prompt string) (string,
|
||||
return result.Candidates[0].Content.Parts[0].Text, nil
|
||||
}
|
||||
|
||||
func (p *geminiProvider) ListModels(_ context.Context) ([]string, error) {
|
||||
return []string{
|
||||
"gemini-2.0-flash",
|
||||
"gemini-2.0-flash-lite",
|
||||
"gemini-1.5-pro",
|
||||
"gemini-1.5-flash",
|
||||
}, nil
|
||||
func (p *geminiProvider) ListModels(ctx context.Context) ([]string, error) {
|
||||
url := fmt.Sprintf("https://generativelanguage.googleapis.com/v1beta/models?key=%s", p.apiKey)
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
resp, err := p.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
raw, _ := io.ReadAll(resp.Body)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("gemini list models error %d: %s", resp.StatusCode, raw)
|
||||
}
|
||||
var result struct {
|
||||
Models []struct {
|
||||
Name string `json:"name"`
|
||||
SupportedMethods []string `json:"supportedGenerationMethods"`
|
||||
} `json:"models"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var names []string
|
||||
for _, m := range result.Models {
|
||||
for _, method := range m.SupportedMethods {
|
||||
if method == "generateContent" {
|
||||
// name is "models/gemini-xxx", strip prefix
|
||||
id := m.Name
|
||||
if len(id) > 7 {
|
||||
id = id[7:] // strip "models/"
|
||||
}
|
||||
names = append(names, id)
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
@ -7,37 +7,62 @@ import (
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"time"
|
||||
)
|
||||
|
||||
type ollamaProvider struct {
|
||||
// OllamaModelInfo holds detailed info about an installed Ollama model.
|
||||
type OllamaModelInfo struct {
|
||||
Name string `json:"name"`
|
||||
Size int64 `json:"size"`
|
||||
ModifiedAt string `json:"modified_at"`
|
||||
Details struct {
|
||||
ParameterSize string `json:"parameter_size"`
|
||||
QuantizationLevel string `json:"quantization_level"`
|
||||
Family string `json:"family"`
|
||||
} `json:"details"`
|
||||
}
|
||||
|
||||
// OllamaProvider implements Provider for Ollama and also exposes model management operations.
|
||||
type OllamaProvider struct {
|
||||
endpoint string
|
||||
model string
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func newOllama(endpoint, model string) *ollamaProvider {
|
||||
func newOllama(endpoint, model string) *OllamaProvider {
|
||||
if endpoint == "" {
|
||||
endpoint = "http://ollama:11434"
|
||||
}
|
||||
if model == "" {
|
||||
model = "llama3"
|
||||
}
|
||||
return &ollamaProvider{
|
||||
return &OllamaProvider{
|
||||
endpoint: endpoint,
|
||||
model: model,
|
||||
client: &http.Client{},
|
||||
}
|
||||
}
|
||||
|
||||
func (p *ollamaProvider) Name() string { return "ollama" }
|
||||
// NewOllamaManager creates an OllamaProvider for model management (pull/delete/list).
|
||||
func NewOllamaManager(endpoint string) *OllamaProvider {
|
||||
return newOllama(endpoint, "")
|
||||
}
|
||||
|
||||
func (p *ollamaProvider) Summarize(ctx context.Context, prompt string) (string, error) {
|
||||
|
||||
func (p *OllamaProvider) Name() string { return "ollama" }
|
||||
|
||||
func (p *OllamaProvider) Summarize(ctx context.Context, prompt string, opts GenOptions) (string, error) {
|
||||
numCtx := 32768
|
||||
if opts.NumCtx > 0 {
|
||||
numCtx = opts.NumCtx
|
||||
}
|
||||
body := map[string]interface{}{
|
||||
"model": p.model,
|
||||
"prompt": prompt,
|
||||
"stream": false,
|
||||
"think": opts.Think,
|
||||
"options": map[string]interface{}{
|
||||
"num_ctx": 32768,
|
||||
"num_ctx": numCtx,
|
||||
},
|
||||
}
|
||||
b, _ := json.Marshal(body)
|
||||
@ -67,7 +92,19 @@ func (p *ollamaProvider) Summarize(ctx context.Context, prompt string) (string,
|
||||
return result.Response, nil
|
||||
}
|
||||
|
||||
func (p *ollamaProvider) ListModels(ctx context.Context) ([]string, error) {
|
||||
func (p *OllamaProvider) ListModels(ctx context.Context) ([]string, error) {
|
||||
infos, err := p.ListModelsDetailed(ctx)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
names := make([]string, len(infos))
|
||||
for i, m := range infos {
|
||||
names[i] = m.Name
|
||||
}
|
||||
return names, nil
|
||||
}
|
||||
|
||||
func (p *OllamaProvider) ListModelsDetailed(ctx context.Context) ([]OllamaModelInfo, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, p.endpoint+"/api/tags", nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@ -80,16 +117,52 @@ func (p *ollamaProvider) ListModels(ctx context.Context) ([]string, error) {
|
||||
|
||||
raw, _ := io.ReadAll(resp.Body)
|
||||
var result struct {
|
||||
Models []struct {
|
||||
Name string `json:"name"`
|
||||
} `json:"models"`
|
||||
Models []OllamaModelInfo `json:"models"`
|
||||
}
|
||||
if err := json.Unmarshal(raw, &result); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
var models []string
|
||||
for _, m := range result.Models {
|
||||
models = append(models, m.Name)
|
||||
return result.Models, nil
|
||||
}
|
||||
return models, nil
|
||||
|
||||
// PullModel pulls (downloads) a model from Ollama Hub. Blocks until complete.
|
||||
func (p *OllamaProvider) PullModel(ctx context.Context, name string) error {
|
||||
body, _ := json.Marshal(map[string]interface{}{"name": name, "stream": false})
|
||||
// Use a long-timeout client since model downloads can take many minutes
|
||||
client := &http.Client{Timeout: 60 * time.Minute}
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, p.endpoint+"/api/pull", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
resp, err := client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
raw, _ := io.ReadAll(resp.Body)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return fmt.Errorf("ollama pull error %d: %s", resp.StatusCode, raw)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
// DeleteModel removes a model from local storage.
|
||||
func (p *OllamaProvider) DeleteModel(ctx context.Context, name string) error {
|
||||
body, _ := json.Marshal(map[string]string{"name": name})
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodDelete, p.endpoint+"/api/delete", bytes.NewReader(body))
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
resp, err := p.client.Do(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != http.StatusOK && resp.StatusCode != http.StatusNoContent {
|
||||
raw, _ := io.ReadAll(resp.Body)
|
||||
return fmt.Errorf("ollama delete error %d: %s", resp.StatusCode, raw)
|
||||
}
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -23,7 +23,7 @@ func newOpenAI(apiKey, model string) *openAIProvider {
|
||||
|
||||
func (p *openAIProvider) Name() string { return "openai" }
|
||||
|
||||
func (p *openAIProvider) Summarize(ctx context.Context, prompt string) (string, error) {
|
||||
func (p *openAIProvider) Summarize(ctx context.Context, prompt string, _ GenOptions) (string, error) {
|
||||
resp, err := p.client.CreateChatCompletion(ctx, openai.ChatCompletionRequest{
|
||||
Model: p.model,
|
||||
Messages: []openai.ChatCompletionMessage{
|
||||
|
||||
@ -3,8 +3,10 @@ package ai
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"regexp"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
"github.com/tradarr/backend/internal/crypto"
|
||||
@ -25,50 +27,53 @@ Structure ton résumé ainsi :
|
||||
type Pipeline struct {
|
||||
repo *models.Repository
|
||||
enc *crypto.Encryptor
|
||||
generating atomic.Bool
|
||||
}
|
||||
|
||||
func NewPipeline(repo *models.Repository, enc *crypto.Encryptor) *Pipeline {
|
||||
return &Pipeline{repo: repo, enc: enc}
|
||||
}
|
||||
|
||||
// BuildProvider instancie un provider à partir de ses paramètres
|
||||
func (p *Pipeline) IsGenerating() bool {
|
||||
return p.generating.Load()
|
||||
}
|
||||
|
||||
func (p *Pipeline) BuildProvider(name, apiKey, endpoint string) (Provider, error) {
|
||||
provider, err := p.repo.GetActiveAIProvider()
|
||||
return NewProvider(name, apiKey, "", endpoint)
|
||||
}
|
||||
|
||||
// buildProviderForRole resolves and builds the AI provider for a given task role.
|
||||
func (p *Pipeline) buildProviderForRole(role string) (Provider, *models.AIProvider, error) {
|
||||
cfg, model, err := p.repo.GetRoleProvider(role)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("get provider for role %s: %w", role, err)
|
||||
}
|
||||
if cfg == nil {
|
||||
return nil, nil, fmt.Errorf("no AI provider configured for role %s", role)
|
||||
}
|
||||
apiKey := ""
|
||||
if cfg.APIKeyEncrypted != "" {
|
||||
apiKey, err = p.enc.Decrypt(cfg.APIKeyEncrypted)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("decrypt API key for role %s: %w", role, err)
|
||||
}
|
||||
}
|
||||
provider, err := NewProvider(cfg.Name, apiKey, model, cfg.Endpoint)
|
||||
if err != nil {
|
||||
return nil, nil, fmt.Errorf("build provider for role %s: %w", role, err)
|
||||
}
|
||||
return provider, cfg, nil
|
||||
}
|
||||
|
||||
func (p *Pipeline) GenerateForUser(ctx context.Context, userID string) (*models.Summary, error) {
|
||||
p.generating.Store(true)
|
||||
defer p.generating.Store(false)
|
||||
|
||||
provider, providerCfg, err := p.buildProviderForRole("summary")
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
model := ""
|
||||
if provider != nil {
|
||||
model = provider.Model
|
||||
}
|
||||
return NewProvider(name, apiKey, model, endpoint)
|
||||
}
|
||||
|
||||
// GenerateForUser génère un résumé personnalisé pour un utilisateur
|
||||
func (p *Pipeline) GenerateForUser(ctx context.Context, userID string) (*models.Summary, error) {
|
||||
// Récupérer le provider actif
|
||||
providerCfg, err := p.repo.GetActiveAIProvider()
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get active provider: %w", err)
|
||||
}
|
||||
if providerCfg == nil {
|
||||
return nil, fmt.Errorf("no active AI provider configured")
|
||||
}
|
||||
|
||||
apiKey := ""
|
||||
if providerCfg.APIKeyEncrypted != "" {
|
||||
apiKey, err = p.enc.Decrypt(providerCfg.APIKeyEncrypted)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("decrypt API key: %w", err)
|
||||
}
|
||||
}
|
||||
|
||||
provider, err := NewProvider(providerCfg.Name, apiKey, providerCfg.Model, providerCfg.Endpoint)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("build provider: %w", err)
|
||||
}
|
||||
|
||||
// Récupérer la watchlist de l'utilisateur (pour le contexte IA uniquement)
|
||||
assets, err := p.repo.GetUserAssets(userID)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("get user assets: %w", err)
|
||||
@ -78,7 +83,6 @@ func (p *Pipeline) GenerateForUser(ctx context.Context, userID string) (*models.
|
||||
symbols[i] = a.Symbol
|
||||
}
|
||||
|
||||
// Récupérer TOUS les articles récents, toutes sources confondues
|
||||
hoursStr, _ := p.repo.GetSetting("articles_lookback_hours")
|
||||
hours, _ := strconv.Atoi(hoursStr)
|
||||
if hours == 0 {
|
||||
@ -98,25 +102,150 @@ func (p *Pipeline) GenerateForUser(ctx context.Context, userID string) (*models.
|
||||
if maxArticles == 0 {
|
||||
maxArticles = 50
|
||||
}
|
||||
if len(articles) > maxArticles {
|
||||
|
||||
// Passe 1 : filtrage par pertinence — seulement si nettement plus d'articles que le max
|
||||
if len(articles) > maxArticles*2 {
|
||||
filterProvider, _, filterErr := p.buildProviderForRole("filter")
|
||||
if filterErr != nil {
|
||||
filterProvider = provider // fallback to summary provider
|
||||
}
|
||||
fmt.Printf("[pipeline] Passe 1 — filtrage : %d articles → sélection des %d plus pertinents…\n", len(articles), maxArticles)
|
||||
t1 := time.Now()
|
||||
articles = p.filterByRelevance(ctx, filterProvider, symbols, articles, maxArticles)
|
||||
fmt.Printf("[pipeline] Passe 1 — terminée en %s : %d articles retenus\n", time.Since(t1).Round(time.Second), len(articles))
|
||||
} else if len(articles) > maxArticles {
|
||||
articles = articles[:maxArticles]
|
||||
fmt.Printf("[pipeline] troncature directe à %d articles (pas assez d'excédent pour justifier un appel IA)\n", maxArticles)
|
||||
}
|
||||
|
||||
systemPrompt, _ := p.repo.GetSetting("ai_system_prompt")
|
||||
if systemPrompt == "" {
|
||||
systemPrompt = DefaultSystemPrompt
|
||||
}
|
||||
prompt := buildPrompt(systemPrompt, symbols, articles)
|
||||
|
||||
summary, err := provider.Summarize(ctx, prompt)
|
||||
tz, _ := p.repo.GetSetting("timezone")
|
||||
|
||||
// Passe 2 : résumé complet
|
||||
fmt.Printf("[pipeline] Passe 2 — résumé : génération sur %d articles…\n", len(articles))
|
||||
t2 := time.Now()
|
||||
prompt := buildPrompt(systemPrompt, symbols, articles, tz)
|
||||
// Passe 2 : think activé pour une meilleure qualité d'analyse
|
||||
summary, err := provider.Summarize(ctx, prompt, GenOptions{Think: true, NumCtx: 32768})
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("AI summarize: %w", err)
|
||||
}
|
||||
fmt.Printf("[pipeline] Passe 2 — terminée en %s\n", time.Since(t2).Round(time.Second))
|
||||
|
||||
return p.repo.CreateSummary(userID, summary, &providerCfg.ID)
|
||||
}
|
||||
|
||||
// GenerateForAll génère les résumés pour tous les utilisateurs ayant une watchlist
|
||||
// filterByRelevance splits articles into batches and asks the AI to select relevant
|
||||
// ones from each batch. Results are pooled then truncated to max.
|
||||
func (p *Pipeline) filterByRelevance(ctx context.Context, provider Provider, symbols []string, articles []models.Article, max int) []models.Article {
|
||||
batchSizeStr, _ := p.repo.GetSetting("filter_batch_size")
|
||||
batchSize, _ := strconv.Atoi(batchSizeStr)
|
||||
if batchSize <= 0 {
|
||||
batchSize = 20
|
||||
}
|
||||
|
||||
var selected []models.Article
|
||||
numBatches := (len(articles) + batchSize - 1) / batchSize
|
||||
|
||||
for b := 0; b < numBatches; b++ {
|
||||
start := b * batchSize
|
||||
end := start + batchSize
|
||||
if end > len(articles) {
|
||||
end = len(articles)
|
||||
}
|
||||
batch := articles[start:end]
|
||||
|
||||
fmt.Printf("[pipeline] Passe 1 — batch %d/%d (%d articles)…\n", b+1, numBatches, len(batch))
|
||||
t := time.Now()
|
||||
chosen := p.filterBatch(ctx, provider, symbols, batch)
|
||||
fmt.Printf("[pipeline] Passe 1 — batch %d/%d terminé en %s : %d retenus\n", b+1, numBatches, time.Since(t).Round(time.Second), len(chosen))
|
||||
|
||||
selected = append(selected, chosen...)
|
||||
|
||||
// Stop early if we have plenty of candidates
|
||||
if len(selected) >= max*2 {
|
||||
fmt.Printf("[pipeline] Passe 1 — suffisamment de candidats (%d), arrêt anticipé\n", len(selected))
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
if len(selected) <= max {
|
||||
return selected
|
||||
}
|
||||
return selected[:max]
|
||||
}
|
||||
|
||||
// filterBatch asks the AI to return all relevant articles from a single batch.
|
||||
func (p *Pipeline) filterBatch(ctx context.Context, provider Provider, symbols []string, batch []models.Article) []models.Article {
|
||||
prompt := buildFilterBatchPrompt(symbols, batch)
|
||||
response, err := provider.Summarize(ctx, prompt, GenOptions{Think: false, NumCtx: 4096})
|
||||
if err != nil {
|
||||
fmt.Printf("[pipeline] filterBatch — échec (%v), conservation du batch entier\n", err)
|
||||
return batch
|
||||
}
|
||||
|
||||
indices := parseIndexArray(response, len(batch))
|
||||
if len(indices) == 0 {
|
||||
return nil
|
||||
}
|
||||
|
||||
filtered := make([]models.Article, 0, len(indices))
|
||||
for _, i := range indices {
|
||||
filtered = append(filtered, batch[i])
|
||||
}
|
||||
return filtered
|
||||
}
|
||||
|
||||
func buildFilterBatchPrompt(symbols []string, batch []models.Article) string {
|
||||
var sb strings.Builder
|
||||
sb.WriteString("Tu es un assistant de trading financier.\n")
|
||||
sb.WriteString(fmt.Sprintf("Parmi les %d articles ci-dessous, sélectionne TOUS ceux pertinents pour un trader actif.\n", len(batch)))
|
||||
|
||||
if len(symbols) > 0 {
|
||||
sb.WriteString("Actifs prioritaires : ")
|
||||
sb.WriteString(strings.Join(symbols, ", "))
|
||||
sb.WriteString("\n")
|
||||
}
|
||||
|
||||
sb.WriteString("\nRéponds UNIQUEMENT avec un tableau JSON des indices retenus (base 0), exemple : [0, 2, 5]\n")
|
||||
sb.WriteString("Si aucun article n'est pertinent, réponds : []\n")
|
||||
sb.WriteString("N'ajoute aucun texte avant ou après le tableau JSON.\n\n")
|
||||
sb.WriteString("Articles :\n")
|
||||
|
||||
for i, a := range batch {
|
||||
sb.WriteString(fmt.Sprintf("[%d] %s (%s)\n", i, a.Title, a.SourceName))
|
||||
}
|
||||
|
||||
return sb.String()
|
||||
}
|
||||
|
||||
var jsonArrayRe = regexp.MustCompile(`\[[\d\s,]+\]`)
|
||||
|
||||
func parseIndexArray(response string, maxIndex int) []int {
|
||||
match := jsonArrayRe.FindString(response)
|
||||
if match == "" {
|
||||
return nil
|
||||
}
|
||||
match = strings.Trim(match, "[]")
|
||||
parts := strings.Split(match, ",")
|
||||
|
||||
seen := make(map[int]bool)
|
||||
var indices []int
|
||||
for _, p := range parts {
|
||||
n, err := strconv.Atoi(strings.TrimSpace(p))
|
||||
if err != nil || n < 0 || n >= maxIndex || seen[n] {
|
||||
continue
|
||||
}
|
||||
seen[n] = true
|
||||
indices = append(indices, n)
|
||||
}
|
||||
return indices
|
||||
}
|
||||
|
||||
func (p *Pipeline) GenerateForAll(ctx context.Context) error {
|
||||
users, err := p.repo.ListUsers()
|
||||
if err != nil {
|
||||
@ -130,7 +259,43 @@ func (p *Pipeline) GenerateForAll(ctx context.Context) error {
|
||||
return nil
|
||||
}
|
||||
|
||||
func buildPrompt(systemPrompt string, symbols []string, articles []models.Article) string {
|
||||
// GenerateReportAsync crée le rapport en DB (status=generating) et lance la génération en arrière-plan.
|
||||
func (p *Pipeline) GenerateReportAsync(reportID, excerpt, question string, mgr *ReportManager) {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)
|
||||
mgr.Register(reportID, cancel)
|
||||
|
||||
go func() {
|
||||
defer cancel()
|
||||
defer mgr.Remove(reportID)
|
||||
|
||||
answer, err := p.callProviderForReport(ctx, excerpt, question)
|
||||
if err != nil {
|
||||
if ctx.Err() != nil {
|
||||
// annulé volontairement — le rapport est supprimé par le handler
|
||||
return
|
||||
}
|
||||
_ = p.repo.UpdateReport(reportID, "error", "", err.Error())
|
||||
return
|
||||
}
|
||||
_ = p.repo.UpdateReport(reportID, "done", answer, "")
|
||||
}()
|
||||
}
|
||||
|
||||
func (p *Pipeline) callProviderForReport(ctx context.Context, excerpt, question string) (string, error) {
|
||||
provider, _, err := p.buildProviderForRole("report")
|
||||
if err != nil {
|
||||
return "", err
|
||||
}
|
||||
|
||||
prompt := fmt.Sprintf(
|
||||
"Tu es un assistant financier expert. L'utilisateur a sélectionné les extraits suivants d'un résumé de marché :\n\n%s\n\nQuestion de l'utilisateur : %s\n\nRéponds en français, de façon précise et orientée trading.",
|
||||
excerpt, question,
|
||||
)
|
||||
|
||||
return provider.Summarize(ctx, prompt, GenOptions{Think: true, NumCtx: 16384})
|
||||
}
|
||||
|
||||
func buildPrompt(systemPrompt string, symbols []string, articles []models.Article, tz string) string {
|
||||
var sb strings.Builder
|
||||
sb.WriteString(systemPrompt)
|
||||
sb.WriteString("\n\n")
|
||||
@ -139,7 +304,11 @@ func buildPrompt(systemPrompt string, symbols []string, articles []models.Articl
|
||||
sb.WriteString(strings.Join(symbols, ", "))
|
||||
sb.WriteString(".\n\n")
|
||||
}
|
||||
sb.WriteString(fmt.Sprintf("Date d'analyse : %s\n\n", time.Now().Format("02/01/2006 15:04")))
|
||||
loc, err := time.LoadLocation(tz)
|
||||
if err != nil || tz == "" {
|
||||
loc = time.UTC
|
||||
}
|
||||
sb.WriteString(fmt.Sprintf("Date d'analyse : %s\n\n", time.Now().In(loc).Format("02/01/2006 15:04")))
|
||||
sb.WriteString("## Actualités\n\n")
|
||||
|
||||
for i, a := range articles {
|
||||
|
||||
@ -5,9 +5,15 @@ import (
|
||||
"fmt"
|
||||
)
|
||||
|
||||
// GenOptions permet de contrôler le comportement de génération par appel.
|
||||
type GenOptions struct {
|
||||
Think bool // active le mode raisonnement (Qwen3 /think)
|
||||
NumCtx int // taille du contexte KV (0 = valeur par défaut du provider)
|
||||
}
|
||||
|
||||
type Provider interface {
|
||||
Name() string
|
||||
Summarize(ctx context.Context, prompt string) (string, error)
|
||||
Summarize(ctx context.Context, prompt string, opts GenOptions) (string, error)
|
||||
ListModels(ctx context.Context) ([]string, error)
|
||||
}
|
||||
|
||||
@ -21,6 +27,8 @@ func NewProvider(name, apiKey, model, endpoint string) (Provider, error) {
|
||||
return newGemini(apiKey, model), nil
|
||||
case "ollama":
|
||||
return newOllama(endpoint, model), nil
|
||||
case "claudecode":
|
||||
return newClaudeCode(model), nil
|
||||
default:
|
||||
return nil, fmt.Errorf("unknown provider: %s", name)
|
||||
}
|
||||
|
||||
37
backend/internal/ai/report_manager.go
Normal file
37
backend/internal/ai/report_manager.go
Normal file
@ -0,0 +1,37 @@
|
||||
package ai
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// ReportManager tracks in-flight report goroutines so they can be cancelled.
|
||||
type ReportManager struct {
|
||||
mu sync.Mutex
|
||||
cancels map[string]context.CancelFunc
|
||||
}
|
||||
|
||||
func NewReportManager() *ReportManager {
|
||||
return &ReportManager{cancels: make(map[string]context.CancelFunc)}
|
||||
}
|
||||
|
||||
func (m *ReportManager) Register(id string, cancel context.CancelFunc) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.cancels[id] = cancel
|
||||
}
|
||||
|
||||
func (m *ReportManager) Cancel(id string) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
if cancel, ok := m.cancels[id]; ok {
|
||||
cancel()
|
||||
delete(m.cancels, id)
|
||||
}
|
||||
}
|
||||
|
||||
func (m *ReportManager) Remove(id string) {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
delete(m.cancels, id)
|
||||
}
|
||||
@ -1,6 +1,7 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"net/http"
|
||||
|
||||
@ -173,6 +174,25 @@ func (h *Handler) DeleteAIProvider(c *gin.Context) {
|
||||
httputil.NoContent(c)
|
||||
}
|
||||
|
||||
func (h *Handler) ProbeAIModels(c *gin.Context) {
|
||||
var req aiProviderRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
httputil.BadRequest(c, err)
|
||||
return
|
||||
}
|
||||
provider, err := h.pipeline.BuildProvider(req.Name, req.APIKey, req.Endpoint)
|
||||
if err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
models, err := provider.ListModels(c.Request.Context())
|
||||
if err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
httputil.OK(c, models)
|
||||
}
|
||||
|
||||
func (h *Handler) ListAIModels(c *gin.Context) {
|
||||
id := c.Param("id")
|
||||
p, err := h.repo.GetAIProviderByID(id)
|
||||
@ -287,10 +307,161 @@ func (h *Handler) UpdateSettings(c *gin.Context) {
|
||||
httputil.OK(c, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
// ── Schedule ───────────────────────────────────────────────────────────────
|
||||
|
||||
func (h *Handler) GetSchedule(c *gin.Context) {
|
||||
slots, err := h.repo.ListScheduleSlots()
|
||||
if err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
httputil.OK(c, slots)
|
||||
}
|
||||
|
||||
type scheduleRequest struct {
|
||||
Slots []struct {
|
||||
DayOfWeek int `json:"day_of_week"`
|
||||
Hour int `json:"hour"`
|
||||
Minute int `json:"minute"`
|
||||
} `json:"slots"`
|
||||
}
|
||||
|
||||
func (h *Handler) UpdateSchedule(c *gin.Context) {
|
||||
var req scheduleRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
httputil.BadRequest(c, err)
|
||||
return
|
||||
}
|
||||
slots := make([]models.ScheduleSlot, len(req.Slots))
|
||||
for i, s := range req.Slots {
|
||||
slots[i] = models.ScheduleSlot{DayOfWeek: s.DayOfWeek, Hour: s.Hour, Minute: s.Minute}
|
||||
}
|
||||
if err := h.repo.ReplaceSchedule(slots); err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
if err := h.scheduler.Reload(); err != nil {
|
||||
fmt.Printf("schedule reload: %v\n", err)
|
||||
}
|
||||
httputil.OK(c, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
func (h *Handler) GetDefaultSystemPrompt(c *gin.Context) {
|
||||
httputil.OK(c, gin.H{"prompt": ai.DefaultSystemPrompt})
|
||||
}
|
||||
|
||||
// ── AI Roles ───────────────────────────────────────────────────────────────
|
||||
|
||||
func (h *Handler) GetAIRoles(c *gin.Context) {
|
||||
roles := []string{"summary", "report", "filter"}
|
||||
resp := gin.H{}
|
||||
for _, role := range roles {
|
||||
providerID, _ := h.repo.GetSetting("ai_role_" + role + "_provider")
|
||||
model, _ := h.repo.GetSetting("ai_role_" + role + "_model")
|
||||
resp[role] = gin.H{"provider_id": providerID, "model": model}
|
||||
}
|
||||
httputil.OK(c, resp)
|
||||
}
|
||||
|
||||
type updateAIRoleRequest struct {
|
||||
ProviderID string `json:"provider_id"`
|
||||
Model string `json:"model"`
|
||||
}
|
||||
|
||||
func (h *Handler) UpdateAIRole(c *gin.Context) {
|
||||
role := c.Param("role")
|
||||
if role != "summary" && role != "report" && role != "filter" {
|
||||
httputil.BadRequest(c, fmt.Errorf("invalid role: must be summary, report, or filter"))
|
||||
return
|
||||
}
|
||||
var req updateAIRoleRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
httputil.BadRequest(c, err)
|
||||
return
|
||||
}
|
||||
if err := h.repo.SetRoleProvider(role, req.ProviderID, req.Model); err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
httputil.OK(c, gin.H{"ok": true})
|
||||
}
|
||||
|
||||
// ── Ollama Model Management ────────────────────────────────────────────────
|
||||
|
||||
func (h *Handler) getOllamaManager() (*ai.OllamaProvider, error) {
|
||||
providers, err := h.repo.ListAIProviders()
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
endpoint := ""
|
||||
for _, p := range providers {
|
||||
if p.Name == "ollama" {
|
||||
endpoint = p.Endpoint
|
||||
break
|
||||
}
|
||||
}
|
||||
if endpoint == "" {
|
||||
endpoint = "http://ollama:11434"
|
||||
}
|
||||
return ai.NewOllamaManager(endpoint), nil
|
||||
}
|
||||
|
||||
func (h *Handler) ListOllamaModels(c *gin.Context) {
|
||||
mgr, err := h.getOllamaManager()
|
||||
if err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
models, err := mgr.ListModelsDetailed(c.Request.Context())
|
||||
if err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
if models == nil {
|
||||
models = []ai.OllamaModelInfo{}
|
||||
}
|
||||
httputil.OK(c, models)
|
||||
}
|
||||
|
||||
type pullModelRequest struct {
|
||||
Name string `json:"name" binding:"required"`
|
||||
}
|
||||
|
||||
func (h *Handler) PullOllamaModel(c *gin.Context) {
|
||||
var req pullModelRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
httputil.BadRequest(c, err)
|
||||
return
|
||||
}
|
||||
mgr, err := h.getOllamaManager()
|
||||
if err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
go func() {
|
||||
if err := mgr.PullModel(context.Background(), req.Name); err != nil {
|
||||
fmt.Printf("[ollama] pull %s failed: %v\n", req.Name, err)
|
||||
} else {
|
||||
fmt.Printf("[ollama] pull %s completed\n", req.Name)
|
||||
}
|
||||
}()
|
||||
c.JSON(202, gin.H{"ok": true, "message": "pull started"})
|
||||
}
|
||||
|
||||
func (h *Handler) DeleteOllamaModel(c *gin.Context) {
|
||||
name := c.Param("name")
|
||||
mgr, err := h.getOllamaManager()
|
||||
if err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
if err := mgr.DeleteModel(c.Request.Context(), name); err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
httputil.NoContent(c)
|
||||
}
|
||||
|
||||
// ── Admin Users ────────────────────────────────────────────────────────────
|
||||
|
||||
func (h *Handler) ListUsers(c *gin.Context) {
|
||||
|
||||
@ -5,6 +5,7 @@ import (
|
||||
"github.com/tradarr/backend/internal/config"
|
||||
"github.com/tradarr/backend/internal/crypto"
|
||||
"github.com/tradarr/backend/internal/models"
|
||||
"github.com/tradarr/backend/internal/scheduler"
|
||||
"github.com/tradarr/backend/internal/scraper"
|
||||
)
|
||||
|
||||
@ -14,6 +15,8 @@ type Handler struct {
|
||||
enc *crypto.Encryptor
|
||||
registry *scraper.Registry
|
||||
pipeline *ai.Pipeline
|
||||
scheduler *scheduler.Scheduler
|
||||
reportManager *ai.ReportManager
|
||||
}
|
||||
|
||||
func New(
|
||||
@ -22,6 +25,7 @@ func New(
|
||||
enc *crypto.Encryptor,
|
||||
registry *scraper.Registry,
|
||||
pipeline *ai.Pipeline,
|
||||
sched *scheduler.Scheduler,
|
||||
) *Handler {
|
||||
return &Handler{
|
||||
repo: repo,
|
||||
@ -29,5 +33,7 @@ func New(
|
||||
enc: enc,
|
||||
registry: registry,
|
||||
pipeline: pipeline,
|
||||
scheduler: sched,
|
||||
reportManager: ai.NewReportManager(),
|
||||
}
|
||||
}
|
||||
|
||||
84
backend/internal/api/handlers/reports.go
Normal file
84
backend/internal/api/handlers/reports.go
Normal file
@ -0,0 +1,84 @@
|
||||
package handlers
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"strings"
|
||||
|
||||
"github.com/gin-gonic/gin"
|
||||
"github.com/tradarr/backend/internal/httputil"
|
||||
)
|
||||
|
||||
type reportRequest struct {
|
||||
SummaryID string `json:"summary_id"`
|
||||
Excerpts []string `json:"excerpts" binding:"required,min=1"`
|
||||
Question string `json:"question" binding:"required"`
|
||||
}
|
||||
|
||||
func (h *Handler) CreateReport(c *gin.Context) {
|
||||
userID := c.GetString("userID")
|
||||
var req reportRequest
|
||||
if err := c.ShouldBindJSON(&req); err != nil {
|
||||
httputil.BadRequest(c, err)
|
||||
return
|
||||
}
|
||||
|
||||
var summaryID *string
|
||||
if req.SummaryID != "" {
|
||||
summaryID = &req.SummaryID
|
||||
}
|
||||
|
||||
// Joindre les extraits avec un séparateur visuel
|
||||
excerpt := buildExcerptContext(req.Excerpts)
|
||||
|
||||
// Créer le rapport en DB avec status=generating, retourner immédiatement
|
||||
report, err := h.repo.CreatePendingReport(userID, summaryID, excerpt, req.Question)
|
||||
if err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
|
||||
// Lancer la génération en arrière-plan
|
||||
h.pipeline.GenerateReportAsync(report.ID, excerpt, req.Question, h.reportManager)
|
||||
|
||||
c.JSON(http.StatusCreated, report)
|
||||
}
|
||||
|
||||
func (h *Handler) ListReports(c *gin.Context) {
|
||||
userID := c.GetString("userID")
|
||||
reports, err := h.repo.ListReports(userID)
|
||||
if err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
httputil.OK(c, reports)
|
||||
}
|
||||
|
||||
func (h *Handler) DeleteReport(c *gin.Context) {
|
||||
userID := c.GetString("userID")
|
||||
id := c.Param("id")
|
||||
// Annuler la goroutine si elle tourne encore
|
||||
h.reportManager.Cancel(id)
|
||||
if err := h.repo.DeleteReport(id, userID); err != nil {
|
||||
httputil.InternalError(c, err)
|
||||
return
|
||||
}
|
||||
c.Status(http.StatusNoContent)
|
||||
}
|
||||
|
||||
func (h *Handler) GetGeneratingStatus(c *gin.Context) {
|
||||
httputil.OK(c, gin.H{"generating": h.pipeline.IsGenerating()})
|
||||
}
|
||||
|
||||
func buildExcerptContext(excerpts []string) string {
|
||||
if len(excerpts) == 1 {
|
||||
return excerpts[0]
|
||||
}
|
||||
var sb strings.Builder
|
||||
for i, e := range excerpts {
|
||||
if i > 0 {
|
||||
sb.WriteString("\n\n---\n\n")
|
||||
}
|
||||
sb.WriteString(e)
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
@ -7,7 +7,11 @@ import (
|
||||
)
|
||||
|
||||
func SetupRouter(h *handlers.Handler, jwtSecret string) *gin.Engine {
|
||||
r := gin.Default()
|
||||
r := gin.New()
|
||||
r.Use(gin.Recovery())
|
||||
r.Use(gin.LoggerWithConfig(gin.LoggerConfig{
|
||||
SkipPaths: []string{"/api/summaries/status"},
|
||||
}))
|
||||
|
||||
r.Use(func(c *gin.Context) {
|
||||
c.Header("Access-Control-Allow-Origin", "*")
|
||||
@ -39,8 +43,13 @@ func SetupRouter(h *handlers.Handler, jwtSecret string) *gin.Engine {
|
||||
authed.GET("/articles/:id", h.GetArticle)
|
||||
|
||||
authed.GET("/summaries", h.ListSummaries)
|
||||
authed.GET("/summaries/status", h.GetGeneratingStatus)
|
||||
authed.POST("/summaries/generate", h.GenerateSummary)
|
||||
|
||||
authed.GET("/reports", h.ListReports)
|
||||
authed.POST("/reports", h.CreateReport)
|
||||
authed.DELETE("/reports/:id", h.DeleteReport)
|
||||
|
||||
// Admin
|
||||
admin := authed.Group("/admin")
|
||||
admin.Use(auth.AdminOnly())
|
||||
@ -54,6 +63,7 @@ func SetupRouter(h *handlers.Handler, jwtSecret string) *gin.Engine {
|
||||
admin.POST("/ai-providers/:id/activate", h.SetActiveAIProvider)
|
||||
admin.DELETE("/ai-providers/:id", h.DeleteAIProvider)
|
||||
admin.GET("/ai-providers/:id/models", h.ListAIModels)
|
||||
admin.POST("/ai-providers/probe-models", h.ProbeAIModels)
|
||||
|
||||
admin.GET("/sources", h.ListSources)
|
||||
admin.PUT("/sources/:id", h.UpdateSource)
|
||||
@ -65,9 +75,21 @@ func SetupRouter(h *handlers.Handler, jwtSecret string) *gin.Engine {
|
||||
admin.PUT("/settings", h.UpdateSettings)
|
||||
admin.GET("/settings/default-prompt", h.GetDefaultSystemPrompt)
|
||||
|
||||
admin.GET("/schedule", h.GetSchedule)
|
||||
admin.PUT("/schedule", h.UpdateSchedule)
|
||||
|
||||
admin.GET("/users", h.ListUsers)
|
||||
admin.PUT("/users/:id", h.UpdateAdminUser)
|
||||
admin.DELETE("/users/:id", h.DeleteAdminUser)
|
||||
|
||||
// AI roles (per-task model assignment)
|
||||
admin.GET("/ai-roles", h.GetAIRoles)
|
||||
admin.PUT("/ai-roles/:role", h.UpdateAIRole)
|
||||
|
||||
// Ollama model management
|
||||
admin.GET("/ollama/models", h.ListOllamaModels)
|
||||
admin.POST("/ollama/pull", h.PullOllamaModel)
|
||||
admin.DELETE("/ollama/models/:name", h.DeleteOllamaModel)
|
||||
|
||||
return r
|
||||
}
|
||||
|
||||
@ -20,7 +20,7 @@ func GenerateToken(userID, email, role, secret string) (string, error) {
|
||||
Email: email,
|
||||
Role: role,
|
||||
RegisteredClaims: jwt.RegisteredClaims{
|
||||
ExpiresAt: jwt.NewNumericDate(time.Now().Add(24 * time.Hour)),
|
||||
ExpiresAt: jwt.NewNumericDate(time.Now().Add(7 * 24 * time.Hour)),
|
||||
IssuedAt: jwt.NewNumericDate(time.Now()),
|
||||
},
|
||||
}
|
||||
|
||||
@ -11,7 +11,7 @@ type Config struct {
|
||||
JWTSecret string
|
||||
EncryptionKey []byte
|
||||
Port string
|
||||
ChromePath string
|
||||
ScraperURL string
|
||||
AdminEmail string
|
||||
AdminPassword string
|
||||
}
|
||||
@ -41,12 +41,17 @@ func Load() (*Config, error) {
|
||||
port = "8080"
|
||||
}
|
||||
|
||||
scraperURL := os.Getenv("SCRAPER_URL")
|
||||
if scraperURL == "" {
|
||||
scraperURL = "http://scraper:3001"
|
||||
}
|
||||
|
||||
return &Config{
|
||||
DatabaseURL: dbURL,
|
||||
JWTSecret: jwtSecret,
|
||||
EncryptionKey: encKey,
|
||||
Port: port,
|
||||
ChromePath: os.Getenv("CHROME_PATH"),
|
||||
ScraperURL: scraperURL,
|
||||
AdminEmail: os.Getenv("ADMIN_EMAIL"),
|
||||
AdminPassword: os.Getenv("ADMIN_PASSWORD"),
|
||||
}, nil
|
||||
|
||||
@ -21,7 +21,7 @@ CREATE TABLE user_assets (
|
||||
CREATE TABLE sources (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
name TEXT NOT NULL,
|
||||
type TEXT NOT NULL CHECK (type IN ('bloomberg', 'stocktwits')),
|
||||
type TEXT NOT NULL CHECK (type IN ('bloomberg', 'stocktwits', 'reuters', 'watcherguru')),
|
||||
enabled BOOLEAN NOT NULL DEFAULT TRUE,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
@ -97,7 +97,7 @@ CREATE INDEX idx_user_assets_user_id ON user_assets(user_id);
|
||||
-- Sources initiales
|
||||
INSERT INTO sources (name, type, enabled) VALUES
|
||||
('Bloomberg', 'bloomberg', TRUE),
|
||||
('StockTwits', 'stocktwits', TRUE);
|
||||
('Yahoo Finance', 'stocktwits', TRUE);
|
||||
|
||||
-- Paramètres par défaut
|
||||
INSERT INTO settings (key, value) VALUES
|
||||
|
||||
@ -0,0 +1 @@
|
||||
DELETE FROM sources WHERE type IN ('reuters', 'watcherguru');
|
||||
@ -0,0 +1,4 @@
|
||||
INSERT INTO sources (name, type, enabled) VALUES
|
||||
('Reuters', 'reuters', true),
|
||||
('Watcher.Guru', 'watcherguru', true)
|
||||
ON CONFLICT DO NOTHING;
|
||||
@ -0,0 +1 @@
|
||||
DROP TABLE IF EXISTS scrape_schedules;
|
||||
17
backend/internal/database/migrations/000004_schedule.up.sql
Normal file
17
backend/internal/database/migrations/000004_schedule.up.sql
Normal file
@ -0,0 +1,17 @@
|
||||
CREATE TABLE scrape_schedules (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
day_of_week SMALLINT NOT NULL CHECK (day_of_week BETWEEN 0 AND 6),
|
||||
hour SMALLINT NOT NULL CHECK (hour BETWEEN 0 AND 23),
|
||||
minute SMALLINT NOT NULL DEFAULT 0 CHECK (minute BETWEEN 0 AND 59),
|
||||
UNIQUE (day_of_week, hour, minute)
|
||||
);
|
||||
|
||||
-- Planning par défaut : lun-ven à 6h et 15h, week-end à 6h uniquement
|
||||
INSERT INTO scrape_schedules (day_of_week, hour, minute) VALUES
|
||||
(1, 6, 0), (1, 15, 0),
|
||||
(2, 6, 0), (2, 15, 0),
|
||||
(3, 6, 0), (3, 15, 0),
|
||||
(4, 6, 0), (4, 15, 0),
|
||||
(5, 6, 0), (5, 15, 0),
|
||||
(6, 6, 0),
|
||||
(0, 6, 0);
|
||||
@ -0,0 +1 @@
|
||||
DROP TABLE IF EXISTS reports;
|
||||
@ -0,0 +1,9 @@
|
||||
CREATE TABLE reports (
|
||||
id UUID PRIMARY KEY DEFAULT gen_random_uuid(),
|
||||
user_id UUID NOT NULL REFERENCES users(id) ON DELETE CASCADE,
|
||||
summary_id UUID REFERENCES summaries(id) ON DELETE SET NULL,
|
||||
context_excerpt TEXT NOT NULL,
|
||||
question TEXT NOT NULL,
|
||||
answer TEXT NOT NULL,
|
||||
created_at TIMESTAMPTZ NOT NULL DEFAULT NOW()
|
||||
);
|
||||
@ -0,0 +1,4 @@
|
||||
ALTER TABLE reports
|
||||
DROP COLUMN IF EXISTS status,
|
||||
DROP COLUMN IF EXISTS error_msg,
|
||||
ALTER COLUMN answer DROP DEFAULT;
|
||||
@ -0,0 +1,4 @@
|
||||
ALTER TABLE reports
|
||||
ALTER COLUMN answer SET DEFAULT '',
|
||||
ADD COLUMN status VARCHAR(20) NOT NULL DEFAULT 'done',
|
||||
ADD COLUMN error_msg TEXT NOT NULL DEFAULT '';
|
||||
@ -0,0 +1 @@
|
||||
DELETE FROM settings WHERE key = 'timezone';
|
||||
@ -0,0 +1,2 @@
|
||||
INSERT INTO settings (key, value) VALUES ('timezone', 'Europe/Paris')
|
||||
ON CONFLICT (key) DO NOTHING;
|
||||
@ -0,0 +1 @@
|
||||
DELETE FROM settings WHERE key = 'filter_batch_size';
|
||||
@ -0,0 +1,2 @@
|
||||
INSERT INTO settings (key, value) VALUES ('filter_batch_size', '20')
|
||||
ON CONFLICT (key) DO NOTHING;
|
||||
@ -0,0 +1,3 @@
|
||||
ALTER TABLE ai_providers DROP CONSTRAINT IF EXISTS ai_providers_name_check;
|
||||
ALTER TABLE ai_providers ADD CONSTRAINT ai_providers_name_check
|
||||
CHECK (name IN ('openai', 'anthropic', 'gemini', 'ollama'));
|
||||
@ -0,0 +1,3 @@
|
||||
ALTER TABLE ai_providers DROP CONSTRAINT IF EXISTS ai_providers_name_check;
|
||||
ALTER TABLE ai_providers ADD CONSTRAINT ai_providers_name_check
|
||||
CHECK (name IN ('openai', 'anthropic', 'gemini', 'ollama', 'claudecode'));
|
||||
@ -97,3 +97,22 @@ type Setting struct {
|
||||
Key string `json:"key"`
|
||||
Value string `json:"value"`
|
||||
}
|
||||
|
||||
type ScheduleSlot struct {
|
||||
ID string `json:"id"`
|
||||
DayOfWeek int `json:"day_of_week"` // 0=dimanche, 1=lundi ... 6=samedi
|
||||
Hour int `json:"hour"`
|
||||
Minute int `json:"minute"`
|
||||
}
|
||||
|
||||
type Report struct {
|
||||
ID string `json:"id"`
|
||||
UserID string `json:"user_id"`
|
||||
SummaryID *string `json:"summary_id"`
|
||||
ContextExcerpt string `json:"context_excerpt"`
|
||||
Question string `json:"question"`
|
||||
Answer string `json:"answer"`
|
||||
Status string `json:"status"` // generating | done | error
|
||||
ErrorMsg string `json:"error_msg"`
|
||||
CreatedAt time.Time `json:"created_at"`
|
||||
}
|
||||
|
||||
@ -187,20 +187,28 @@ func (r *Repository) UpdateSource(id string, enabled bool) error {
|
||||
|
||||
// ── Articles ───────────────────────────────────────────────────────────────
|
||||
|
||||
func (r *Repository) UpsertArticle(sourceID, title, content, url string, publishedAt *time.Time) (*Article, error) {
|
||||
a := &Article{}
|
||||
// InsertArticleIfNew insère l'article uniquement s'il n'existe pas déjà (par URL).
|
||||
// Retourne (article, true, nil) si inséré, (nil, false, nil) si déjà présent.
|
||||
func (r *Repository) InsertArticleIfNew(sourceID, title, content, url string, publishedAt *time.Time) (*Article, bool, error) {
|
||||
var pa sql.NullTime
|
||||
if publishedAt != nil {
|
||||
pa = sql.NullTime{Time: *publishedAt, Valid: true}
|
||||
}
|
||||
a := &Article{}
|
||||
err := r.db.QueryRow(`
|
||||
INSERT INTO articles (source_id, title, content, url, published_at)
|
||||
VALUES ($1, $2, $3, $4, $5)
|
||||
ON CONFLICT (url) DO UPDATE SET title=EXCLUDED.title, content=EXCLUDED.content
|
||||
ON CONFLICT (url) DO NOTHING
|
||||
RETURNING id, source_id, title, content, url, published_at, created_at`,
|
||||
sourceID, title, content, url, pa,
|
||||
).Scan(&a.ID, &a.SourceID, &a.Title, &a.Content, &a.URL, &a.PublishedAt, &a.CreatedAt)
|
||||
return a, err
|
||||
if err == sql.ErrNoRows {
|
||||
return nil, false, nil // déjà présent
|
||||
}
|
||||
if err != nil {
|
||||
return nil, false, err
|
||||
}
|
||||
return a, true, nil
|
||||
}
|
||||
|
||||
func (r *Repository) AddArticleSymbol(articleID, symbol string) error {
|
||||
@ -260,7 +268,7 @@ func (r *Repository) GetRecentArticles(hours int) ([]Article, error) {
|
||||
SELECT a.id, a.source_id, s.name, a.title, a.content, a.url, a.published_at, a.created_at
|
||||
FROM articles a
|
||||
JOIN sources s ON s.id = a.source_id
|
||||
WHERE a.created_at > NOW() - ($1 * INTERVAL '1 hour')
|
||||
WHERE COALESCE(a.published_at, a.created_at) > NOW() - ($1 * INTERVAL '1 hour')
|
||||
ORDER BY a.published_at DESC NULLS LAST, a.created_at DESC`, hours)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@ -520,6 +528,51 @@ func (r *Repository) SetSetting(key, value string) error {
|
||||
return err
|
||||
}
|
||||
|
||||
// ── Schedule ───────────────────────────────────────────────────────────────
|
||||
|
||||
func (r *Repository) ListScheduleSlots() ([]ScheduleSlot, error) {
|
||||
rows, err := r.db.Query(`
|
||||
SELECT id, day_of_week, hour, minute FROM scrape_schedules
|
||||
ORDER BY day_of_week, hour, minute`)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var slots []ScheduleSlot
|
||||
for rows.Next() {
|
||||
var s ScheduleSlot
|
||||
if err := rows.Scan(&s.ID, &s.DayOfWeek, &s.Hour, &s.Minute); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
slots = append(slots, s)
|
||||
}
|
||||
return slots, nil
|
||||
}
|
||||
|
||||
func (r *Repository) ReplaceSchedule(slots []ScheduleSlot) error {
|
||||
tx, err := r.db.Begin()
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer tx.Rollback()
|
||||
|
||||
if _, err := tx.Exec(`DELETE FROM scrape_schedules`); err != nil {
|
||||
return err
|
||||
}
|
||||
for _, s := range slots {
|
||||
if _, err := tx.Exec(
|
||||
`INSERT INTO scrape_schedules (day_of_week, hour, minute) VALUES ($1, $2, $3)
|
||||
ON CONFLICT (day_of_week, hour, minute) DO NOTHING`,
|
||||
s.DayOfWeek, s.Hour, s.Minute,
|
||||
); err != nil {
|
||||
return err
|
||||
}
|
||||
}
|
||||
return tx.Commit()
|
||||
}
|
||||
|
||||
// ── Settings ───────────────────────────────────────────────────────────────
|
||||
|
||||
func (r *Repository) ListSettings() ([]Setting, error) {
|
||||
rows, err := r.db.Query(`SELECT key, value FROM settings ORDER BY key`)
|
||||
if err != nil {
|
||||
@ -536,3 +589,80 @@ func (r *Repository) ListSettings() ([]Setting, error) {
|
||||
}
|
||||
return settings, nil
|
||||
}
|
||||
|
||||
// ── AI Role Providers ──────────────────────────────────────────────────────
|
||||
|
||||
// GetRoleProvider returns the configured provider and model for a given role (summary/report/filter).
|
||||
// Falls back to the active provider if no role-specific provider is set.
|
||||
func (r *Repository) GetRoleProvider(role string) (*AIProvider, string, error) {
|
||||
providerID, _ := r.GetSetting("ai_role_" + role + "_provider")
|
||||
model, _ := r.GetSetting("ai_role_" + role + "_model")
|
||||
|
||||
if providerID != "" {
|
||||
p, err := r.GetAIProviderByID(providerID)
|
||||
if err == nil && p != nil {
|
||||
if model == "" {
|
||||
model = p.Model
|
||||
}
|
||||
return p, model, nil
|
||||
}
|
||||
}
|
||||
// Fallback to active provider
|
||||
p, err := r.GetActiveAIProvider()
|
||||
if p != nil && model == "" {
|
||||
model = p.Model
|
||||
}
|
||||
return p, model, err
|
||||
}
|
||||
|
||||
func (r *Repository) SetRoleProvider(role, providerID, model string) error {
|
||||
if err := r.SetSetting("ai_role_"+role+"_provider", providerID); err != nil {
|
||||
return err
|
||||
}
|
||||
return r.SetSetting("ai_role_"+role+"_model", model)
|
||||
}
|
||||
|
||||
// ── Reports ────────────────────────────────────────────────────────────────
|
||||
|
||||
func (r *Repository) CreatePendingReport(userID string, summaryID *string, excerpt, question string) (*Report, error) {
|
||||
rep := &Report{}
|
||||
err := r.db.QueryRow(`
|
||||
INSERT INTO reports (user_id, summary_id, context_excerpt, question, answer, status)
|
||||
VALUES ($1, $2, $3, $4, '', 'generating')
|
||||
RETURNING id, user_id, summary_id, context_excerpt, question, answer, status, error_msg, created_at`,
|
||||
userID, summaryID, excerpt, question,
|
||||
).Scan(&rep.ID, &rep.UserID, &rep.SummaryID, &rep.ContextExcerpt, &rep.Question, &rep.Answer, &rep.Status, &rep.ErrorMsg, &rep.CreatedAt)
|
||||
return rep, err
|
||||
}
|
||||
|
||||
func (r *Repository) UpdateReport(id, status, answer, errorMsg string) error {
|
||||
_, err := r.db.Exec(`
|
||||
UPDATE reports SET status=$1, answer=$2, error_msg=$3 WHERE id=$4`,
|
||||
status, answer, errorMsg, id)
|
||||
return err
|
||||
}
|
||||
|
||||
func (r *Repository) ListReports(userID string) ([]Report, error) {
|
||||
rows, err := r.db.Query(`
|
||||
SELECT id, user_id, summary_id, context_excerpt, question, answer, status, error_msg, created_at
|
||||
FROM reports WHERE user_id=$1
|
||||
ORDER BY created_at DESC`, userID)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer rows.Close()
|
||||
var reports []Report
|
||||
for rows.Next() {
|
||||
var rep Report
|
||||
if err := rows.Scan(&rep.ID, &rep.UserID, &rep.SummaryID, &rep.ContextExcerpt, &rep.Question, &rep.Answer, &rep.Status, &rep.ErrorMsg, &rep.CreatedAt); err != nil {
|
||||
return nil, err
|
||||
}
|
||||
reports = append(reports, rep)
|
||||
}
|
||||
return reports, nil
|
||||
}
|
||||
|
||||
func (r *Repository) DeleteReport(id, userID string) error {
|
||||
_, err := r.db.Exec(`DELETE FROM reports WHERE id=$1 AND user_id=$2`, id, userID)
|
||||
return err
|
||||
}
|
||||
|
||||
@ -3,7 +3,6 @@ package scheduler
|
||||
import (
|
||||
"context"
|
||||
"fmt"
|
||||
"strconv"
|
||||
|
||||
"github.com/robfig/cron/v3"
|
||||
"github.com/tradarr/backend/internal/ai"
|
||||
@ -16,7 +15,7 @@ type Scheduler struct {
|
||||
registry *scraper.Registry
|
||||
pipeline *ai.Pipeline
|
||||
repo *models.Repository
|
||||
entryID cron.EntryID
|
||||
entryIDs []cron.EntryID
|
||||
}
|
||||
|
||||
func New(registry *scraper.Registry, pipeline *ai.Pipeline, repo *models.Repository) *Scheduler {
|
||||
@ -29,19 +28,10 @@ func New(registry *scraper.Registry, pipeline *ai.Pipeline, repo *models.Reposit
|
||||
}
|
||||
|
||||
func (s *Scheduler) Start() error {
|
||||
interval, err := s.getInterval()
|
||||
if err != nil {
|
||||
if err := s.loadSchedule(); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
spec := fmt.Sprintf("@every %dm", interval)
|
||||
s.entryID, err = s.cron.AddFunc(spec, s.run)
|
||||
if err != nil {
|
||||
return fmt.Errorf("add cron: %w", err)
|
||||
}
|
||||
|
||||
s.cron.Start()
|
||||
fmt.Printf("scheduler started, running every %d minutes\n", interval)
|
||||
return nil
|
||||
}
|
||||
|
||||
@ -50,39 +40,51 @@ func (s *Scheduler) Stop() {
|
||||
}
|
||||
|
||||
func (s *Scheduler) Reload() error {
|
||||
s.cron.Remove(s.entryID)
|
||||
interval, err := s.getInterval()
|
||||
if err != nil {
|
||||
return err
|
||||
for _, id := range s.entryIDs {
|
||||
s.cron.Remove(id)
|
||||
}
|
||||
spec := fmt.Sprintf("@every %dm", interval)
|
||||
s.entryID, err = s.cron.AddFunc(spec, s.run)
|
||||
return err
|
||||
s.entryIDs = nil
|
||||
return s.loadSchedule()
|
||||
}
|
||||
|
||||
func (s *Scheduler) loadSchedule() error {
|
||||
slots, err := s.repo.ListScheduleSlots()
|
||||
if err != nil {
|
||||
return fmt.Errorf("load schedule: %w", err)
|
||||
}
|
||||
if len(slots) == 0 {
|
||||
fmt.Println("scheduler: no schedule configured, scraping disabled")
|
||||
return nil
|
||||
}
|
||||
|
||||
tz, _ := s.repo.GetSetting("timezone")
|
||||
if tz == "" {
|
||||
tz = "UTC"
|
||||
}
|
||||
|
||||
for _, slot := range slots {
|
||||
// TZ= prefix permet à robfig/cron d'interpréter les heures dans le fuseau configuré
|
||||
spec := fmt.Sprintf("TZ=%s %d %d * * %d", tz, slot.Minute, slot.Hour, slot.DayOfWeek)
|
||||
id, err := s.cron.AddFunc(spec, s.run)
|
||||
if err != nil {
|
||||
fmt.Printf("scheduler: invalid cron spec %q: %v\n", spec, err)
|
||||
continue
|
||||
}
|
||||
s.entryIDs = append(s.entryIDs, id)
|
||||
}
|
||||
|
||||
fmt.Printf("scheduler: %d time slots loaded (timezone: %s)\n", len(s.entryIDs), tz)
|
||||
return nil
|
||||
}
|
||||
|
||||
func (s *Scheduler) run() {
|
||||
fmt.Println("scheduler: running scraping cycle")
|
||||
fmt.Println("scheduler: starting scraping cycle")
|
||||
if err := s.registry.RunAll(); err != nil {
|
||||
fmt.Printf("scheduler scrape error: %v\n", err)
|
||||
return
|
||||
}
|
||||
fmt.Println("scheduler: running AI summaries")
|
||||
fmt.Println("scheduler: starting AI summaries")
|
||||
if err := s.pipeline.GenerateForAll(context.Background()); err != nil {
|
||||
fmt.Printf("scheduler summary error: %v\n", err)
|
||||
}
|
||||
}
|
||||
|
||||
func (s *Scheduler) getInterval() (int, error) {
|
||||
v, err := s.repo.GetSetting("scrape_interval_minutes")
|
||||
if err != nil {
|
||||
return 60, nil
|
||||
}
|
||||
if v == "" {
|
||||
return 60, nil
|
||||
}
|
||||
n, err := strconv.Atoi(v)
|
||||
if err != nil || n < 1 {
|
||||
return 60, nil
|
||||
}
|
||||
return n, nil
|
||||
}
|
||||
|
||||
@ -1,206 +1,94 @@
|
||||
package bloomberg
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/json"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/chromedp/chromedp"
|
||||
"github.com/tradarr/backend/internal/scraper"
|
||||
)
|
||||
|
||||
type Bloomberg struct {
|
||||
username string
|
||||
password string
|
||||
chromePath string
|
||||
scraperURL string
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func New(username, password, chromePath string) *Bloomberg {
|
||||
return &Bloomberg{username: username, password: password, chromePath: chromePath}
|
||||
func New(scraperURL string) *Bloomberg {
|
||||
if scraperURL == "" {
|
||||
scraperURL = "http://scraper:3001"
|
||||
}
|
||||
return &Bloomberg{
|
||||
scraperURL: scraperURL,
|
||||
client: &http.Client{Timeout: 10 * time.Minute},
|
||||
}
|
||||
}
|
||||
|
||||
func (b *Bloomberg) Name() string { return "bloomberg" }
|
||||
|
||||
func (b *Bloomberg) Scrape(ctx context.Context, symbols []string) ([]scraper.Article, error) {
|
||||
if b.username == "" || b.password == "" {
|
||||
return nil, fmt.Errorf("bloomberg credentials not configured")
|
||||
type scraperRequest struct {
|
||||
Username string `json:"username"`
|
||||
Password string `json:"password"`
|
||||
}
|
||||
|
||||
opts := []chromedp.ExecAllocatorOption{
|
||||
chromedp.NoFirstRun,
|
||||
chromedp.NoDefaultBrowserCheck,
|
||||
chromedp.Headless,
|
||||
chromedp.DisableGPU,
|
||||
chromedp.Flag("no-sandbox", true),
|
||||
chromedp.Flag("disable-setuid-sandbox", true),
|
||||
chromedp.Flag("disable-dev-shm-usage", true),
|
||||
chromedp.Flag("disable-blink-features", "AutomationControlled"),
|
||||
chromedp.Flag("disable-infobars", true),
|
||||
chromedp.Flag("window-size", "1920,1080"),
|
||||
chromedp.Flag("ignore-certificate-errors", true),
|
||||
chromedp.UserAgent("Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/122.0.0.0 Safari/537.36"),
|
||||
}
|
||||
if b.chromePath != "" {
|
||||
opts = append(opts, chromedp.ExecPath(b.chromePath))
|
||||
type scraperArticle struct {
|
||||
Title string `json:"title"`
|
||||
URL string `json:"url"`
|
||||
}
|
||||
|
||||
allocCtx, cancelAlloc := chromedp.NewExecAllocator(ctx, opts...)
|
||||
defer cancelAlloc()
|
||||
|
||||
chromeCtx, cancelChrome := chromedp.NewContext(allocCtx)
|
||||
defer cancelChrome()
|
||||
|
||||
timeoutCtx, cancelTimeout := context.WithTimeout(chromeCtx, 5*time.Minute)
|
||||
defer cancelTimeout()
|
||||
|
||||
if err := b.login(timeoutCtx); err != nil {
|
||||
return nil, fmt.Errorf("bloomberg login: %w", err)
|
||||
type scraperResponse struct {
|
||||
Articles []scraperArticle `json:"articles"`
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
|
||||
var articles []scraper.Article
|
||||
pages := []string{
|
||||
"https://www.bloomberg.com/markets",
|
||||
"https://www.bloomberg.com/technology",
|
||||
"https://www.bloomberg.com/economics",
|
||||
}
|
||||
for _, u := range pages {
|
||||
pageArticles, err := b.scrapePage(timeoutCtx, u, symbols)
|
||||
func (b *Bloomberg) ScrapeWithCredentials(ctx context.Context, username, password string, symbols []string) ([]scraper.Article, error) {
|
||||
payload, _ := json.Marshal(scraperRequest{Username: username, Password: password})
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, b.scraperURL+"/bloomberg/scrape", bytes.NewReader(payload))
|
||||
if err != nil {
|
||||
fmt.Printf("bloomberg scrape %s: %v\n", u, err)
|
||||
continue
|
||||
}
|
||||
articles = append(articles, pageArticles...)
|
||||
}
|
||||
fmt.Printf("bloomberg: %d articles fetched total\n", len(articles))
|
||||
return articles, nil
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
|
||||
func (b *Bloomberg) login(ctx context.Context) error {
|
||||
loginCtx, cancel := context.WithTimeout(ctx, 2*time.Minute)
|
||||
defer cancel()
|
||||
|
||||
// Masquer la détection d'automation via JS
|
||||
if err := chromedp.Run(loginCtx,
|
||||
chromedp.ActionFunc(func(ctx context.Context) error {
|
||||
return chromedp.Evaluate(`
|
||||
Object.defineProperty(navigator, 'webdriver', {get: () => undefined});
|
||||
window.chrome = { runtime: {} };
|
||||
`, nil).Do(ctx)
|
||||
}),
|
||||
); err != nil {
|
||||
fmt.Printf("bloomberg: could not inject stealth JS: %v\n", err)
|
||||
}
|
||||
|
||||
err := chromedp.Run(loginCtx,
|
||||
chromedp.Navigate("https://www.bloomberg.com/account/signin"),
|
||||
chromedp.Sleep(2*time.Second),
|
||||
// Essayer plusieurs sélecteurs pour l'email
|
||||
chromedp.ActionFunc(func(ctx context.Context) error {
|
||||
selectors := []string{
|
||||
`input[name="email"]`,
|
||||
`input[type="email"]`,
|
||||
`input[data-type="email"]`,
|
||||
`input[placeholder*="email" i]`,
|
||||
`input[placeholder*="mail" i]`,
|
||||
}
|
||||
for _, sel := range selectors {
|
||||
var count int
|
||||
if err := chromedp.Evaluate(fmt.Sprintf(`document.querySelectorAll('%s').length`, sel), &count).Do(ctx); err == nil && count > 0 {
|
||||
fmt.Printf("bloomberg: using email selector: %s\n", sel)
|
||||
return chromedp.SendKeys(sel, b.username, chromedp.ByQuery).Do(ctx)
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("could not find email input — Bloomberg login page structure may have changed")
|
||||
}),
|
||||
chromedp.Sleep(500*time.Millisecond),
|
||||
// Submit email
|
||||
chromedp.ActionFunc(func(ctx context.Context) error {
|
||||
selectors := []string{`button[type="submit"]`, `input[type="submit"]`, `button[data-testid*="submit"]`}
|
||||
for _, sel := range selectors {
|
||||
var count int
|
||||
if err := chromedp.Evaluate(fmt.Sprintf(`document.querySelectorAll('%s').length`, sel), &count).Do(ctx); err == nil && count > 0 {
|
||||
return chromedp.Click(sel, chromedp.ByQuery).Do(ctx)
|
||||
}
|
||||
}
|
||||
// Fallback: press Enter
|
||||
return chromedp.KeyEvent("\r").Do(ctx)
|
||||
}),
|
||||
chromedp.Sleep(2*time.Second),
|
||||
// Password
|
||||
chromedp.ActionFunc(func(ctx context.Context) error {
|
||||
selectors := []string{`input[type="password"]`, `input[name="password"]`}
|
||||
for _, sel := range selectors {
|
||||
var count int
|
||||
if err := chromedp.Evaluate(fmt.Sprintf(`document.querySelectorAll('%s').length`, sel), &count).Do(ctx); err == nil && count > 0 {
|
||||
fmt.Printf("bloomberg: using password selector: %s\n", sel)
|
||||
return chromedp.SendKeys(sel, b.password, chromedp.ByQuery).Do(ctx)
|
||||
}
|
||||
}
|
||||
return fmt.Errorf("could not find password input")
|
||||
}),
|
||||
chromedp.Sleep(500*time.Millisecond),
|
||||
chromedp.ActionFunc(func(ctx context.Context) error {
|
||||
selectors := []string{`button[type="submit"]`, `input[type="submit"]`}
|
||||
for _, sel := range selectors {
|
||||
var count int
|
||||
if err := chromedp.Evaluate(fmt.Sprintf(`document.querySelectorAll('%s').length`, sel), &count).Do(ctx); err == nil && count > 0 {
|
||||
return chromedp.Click(sel, chromedp.ByQuery).Do(ctx)
|
||||
}
|
||||
}
|
||||
return chromedp.KeyEvent("\r").Do(ctx)
|
||||
}),
|
||||
chromedp.Sleep(3*time.Second),
|
||||
)
|
||||
return err
|
||||
}
|
||||
|
||||
func (b *Bloomberg) scrapePage(ctx context.Context, pageURL string, symbols []string) ([]scraper.Article, error) {
|
||||
pageCtx, cancel := context.WithTimeout(ctx, 60*time.Second)
|
||||
defer cancel()
|
||||
|
||||
var articleNodes []map[string]string
|
||||
err := chromedp.Run(pageCtx,
|
||||
chromedp.Navigate(pageURL),
|
||||
chromedp.Sleep(3*time.Second),
|
||||
chromedp.Evaluate(`
|
||||
(function() {
|
||||
var items = [];
|
||||
var seen = new Set();
|
||||
var links = document.querySelectorAll('a[href*="/news/articles"], a[href*="/opinion/"], a[href*="/markets/"]');
|
||||
links.forEach(function(a) {
|
||||
if (seen.has(a.href)) return;
|
||||
seen.add(a.href);
|
||||
var title = a.querySelector('h1,h2,h3,h4,[class*="headline"],[class*="title"]');
|
||||
var text = title ? title.innerText.trim() : a.innerText.trim();
|
||||
if (text.length > 20 && a.href.includes('bloomberg.com')) {
|
||||
items.push({title: text, url: a.href});
|
||||
}
|
||||
});
|
||||
return items.slice(0, 25);
|
||||
})()
|
||||
`, &articleNodes),
|
||||
)
|
||||
resp, err := b.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("navigate %s: %w", pageURL, err)
|
||||
return nil, fmt.Errorf("scraper service unreachable: %w", err)
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
body, _ := io.ReadAll(resp.Body)
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("scraper service HTTP %d: %s", resp.StatusCode, body)
|
||||
}
|
||||
|
||||
var result scraperResponse
|
||||
if err := json.Unmarshal(body, &result); err != nil {
|
||||
return nil, fmt.Errorf("parse scraper response: %w", err)
|
||||
}
|
||||
if result.Error != "" {
|
||||
return nil, fmt.Errorf("bloomberg: %s", result.Error)
|
||||
}
|
||||
|
||||
var articles []scraper.Article
|
||||
now := time.Now()
|
||||
for _, node := range articleNodes {
|
||||
title := strings.TrimSpace(node["title"])
|
||||
url := node["url"]
|
||||
if title == "" || url == "" || !strings.Contains(url, "bloomberg.com") {
|
||||
var articles []scraper.Article
|
||||
for _, a := range result.Articles {
|
||||
title := strings.TrimSpace(a.Title)
|
||||
url := a.URL
|
||||
if title == "" || url == "" {
|
||||
continue
|
||||
}
|
||||
syms := scraper.DetectSymbols(title, symbols)
|
||||
articles = append(articles, scraper.Article{
|
||||
Title: title,
|
||||
Content: title, // contenu minimal — l'article complet nécessite un accès payant
|
||||
Content: title,
|
||||
URL: url,
|
||||
PublishedAt: &now,
|
||||
Symbols: syms,
|
||||
})
|
||||
}
|
||||
fmt.Printf("bloomberg: %d articles fetched\n", len(articles))
|
||||
return articles, nil
|
||||
}
|
||||
|
||||
@ -9,21 +9,19 @@ import (
|
||||
"github.com/tradarr/backend/internal/scraper"
|
||||
)
|
||||
|
||||
// DynamicBloomberg charge les credentials depuis la DB avant chaque scraping
|
||||
type DynamicBloomberg struct {
|
||||
repo *models.Repository
|
||||
enc *crypto.Encryptor
|
||||
chromePath string
|
||||
scraperURL string
|
||||
}
|
||||
|
||||
func NewDynamic(repo *models.Repository, enc *crypto.Encryptor, chromePath string) *DynamicBloomberg {
|
||||
return &DynamicBloomberg{repo: repo, enc: enc, chromePath: chromePath}
|
||||
func NewDynamic(repo *models.Repository, enc *crypto.Encryptor, scraperURL string) *DynamicBloomberg {
|
||||
return &DynamicBloomberg{repo: repo, enc: enc, scraperURL: scraperURL}
|
||||
}
|
||||
|
||||
func (d *DynamicBloomberg) Name() string { return "bloomberg" }
|
||||
|
||||
func (d *DynamicBloomberg) Scrape(ctx context.Context, symbols []string) ([]scraper.Article, error) {
|
||||
// Récupérer la source Bloomberg
|
||||
source, err := d.repo.GetSourceByType("bloomberg")
|
||||
if err != nil || source == nil {
|
||||
return nil, fmt.Errorf("bloomberg source not found")
|
||||
@ -34,7 +32,7 @@ func (d *DynamicBloomberg) Scrape(ctx context.Context, symbols []string) ([]scra
|
||||
return nil, fmt.Errorf("get bloomberg credentials: %w", err)
|
||||
}
|
||||
if cred == nil || cred.Username == "" {
|
||||
return nil, fmt.Errorf("bloomberg credentials not configured — please set them in the admin panel")
|
||||
return nil, fmt.Errorf("bloomberg credentials not configured — configure them in the admin panel")
|
||||
}
|
||||
|
||||
password := ""
|
||||
@ -45,6 +43,6 @@ func (d *DynamicBloomberg) Scrape(ctx context.Context, symbols []string) ([]scra
|
||||
}
|
||||
}
|
||||
|
||||
b := New(cred.Username, password, d.chromePath)
|
||||
return b.Scrape(ctx, symbols)
|
||||
b := New(d.scraperURL)
|
||||
return b.ScrapeWithCredentials(ctx, cred.Username, password, symbols)
|
||||
}
|
||||
|
||||
@ -72,11 +72,11 @@ func (r *Registry) Run(sourceID string) error {
|
||||
return scrapeErr
|
||||
}
|
||||
|
||||
// Persister les articles
|
||||
// Persister uniquement les nouveaux articles
|
||||
count := 0
|
||||
for _, a := range articles {
|
||||
saved, err := r.repo.UpsertArticle(sourceID, a.Title, a.Content, a.URL, a.PublishedAt)
|
||||
if err != nil {
|
||||
saved, isNew, err := r.repo.InsertArticleIfNew(sourceID, a.Title, a.Content, a.URL, a.PublishedAt)
|
||||
if err != nil || !isNew {
|
||||
continue
|
||||
}
|
||||
count++
|
||||
|
||||
129
backend/internal/scraper/reuters/reuters.go
Normal file
129
backend/internal/scraper/reuters/reuters.go
Normal file
@ -0,0 +1,129 @@
|
||||
package reuters
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/xml"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"github.com/tradarr/backend/internal/scraper"
|
||||
)
|
||||
|
||||
// Reuters RSS est bloqué par Cloudflare. On utilise des flux RSS financiers
|
||||
// publics fiables à la place : MarketWatch, CNBC, Seeking Alpha.
|
||||
var feeds = []struct {
|
||||
name string
|
||||
url string
|
||||
}{
|
||||
{"MarketWatch Top Stories", "https://feeds.content.dowjones.io/public/rss/mw_topstories"},
|
||||
{"MarketWatch Markets", "https://feeds.content.dowjones.io/public/rss/mw_marketpulse"},
|
||||
{"CNBC Top News", "https://search.cnbc.com/rs/search/combinedcombined/rss/topNews"},
|
||||
{"CNBC Finance", "https://search.cnbc.com/rs/search/combinedcombined/rss/topNews?tag=Finance"},
|
||||
}
|
||||
|
||||
type Reuters struct {
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func New() *Reuters {
|
||||
return &Reuters{client: &http.Client{Timeout: 15 * time.Second}}
|
||||
}
|
||||
|
||||
func (r *Reuters) Name() string { return "reuters" }
|
||||
|
||||
type rssFeed struct {
|
||||
Channel struct {
|
||||
Items []struct {
|
||||
Title string `xml:"title"`
|
||||
Link string `xml:"link"`
|
||||
Description string `xml:"description"`
|
||||
PubDate string `xml:"pubDate"`
|
||||
} `xml:"item"`
|
||||
} `xml:"channel"`
|
||||
}
|
||||
|
||||
func (r *Reuters) Scrape(ctx context.Context, _ []string) ([]scraper.Article, error) {
|
||||
var articles []scraper.Article
|
||||
seen := make(map[string]bool)
|
||||
|
||||
for i, feed := range feeds {
|
||||
if i > 0 {
|
||||
select {
|
||||
case <-ctx.Done():
|
||||
return articles, ctx.Err()
|
||||
case <-time.After(300 * time.Millisecond):
|
||||
}
|
||||
}
|
||||
items, err := r.fetchFeed(ctx, feed.url)
|
||||
if err != nil {
|
||||
fmt.Printf("reuters/financial %s: %v\n", feed.name, err)
|
||||
continue
|
||||
}
|
||||
for _, a := range items {
|
||||
if !seen[a.URL] {
|
||||
seen[a.URL] = true
|
||||
articles = append(articles, a)
|
||||
}
|
||||
}
|
||||
fmt.Printf("reuters/financial %s: %d articles\n", feed.name, len(items))
|
||||
}
|
||||
return articles, nil
|
||||
}
|
||||
|
||||
func (r *Reuters) fetchFeed(ctx context.Context, feedURL string) ([]scraper.Article, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, feedURL, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("User-Agent", "Mozilla/5.0 (compatible; Tradarr/1.0)")
|
||||
req.Header.Set("Accept", "application/rss+xml, application/xml, text/xml")
|
||||
|
||||
resp, err := r.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
body, _ := io.ReadAll(io.LimitReader(resp.Body, 256))
|
||||
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, body)
|
||||
}
|
||||
|
||||
var feed rssFeed
|
||||
if err := xml.NewDecoder(resp.Body).Decode(&feed); err != nil {
|
||||
return nil, fmt.Errorf("parse RSS: %w", err)
|
||||
}
|
||||
|
||||
var articles []scraper.Article
|
||||
for _, item := range feed.Channel.Items {
|
||||
title := strings.TrimSpace(item.Title)
|
||||
link := strings.TrimSpace(item.Link)
|
||||
if title == "" || link == "" {
|
||||
continue
|
||||
}
|
||||
|
||||
var publishedAt *time.Time
|
||||
for _, f := range []string{time.RFC1123Z, time.RFC1123, "Mon, 02 Jan 2006 15:04:05 -0700"} {
|
||||
if t, err := time.Parse(f, item.PubDate); err == nil {
|
||||
publishedAt = &t
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
content := strings.TrimSpace(item.Description)
|
||||
if content == "" {
|
||||
content = title
|
||||
}
|
||||
|
||||
articles = append(articles, scraper.Article{
|
||||
Title: title,
|
||||
Content: content,
|
||||
URL: link,
|
||||
PublishedAt: publishedAt,
|
||||
})
|
||||
}
|
||||
return articles, nil
|
||||
}
|
||||
200
backend/internal/scraper/watcherguru/watcherguru.go
Normal file
200
backend/internal/scraper/watcherguru/watcherguru.go
Normal file
@ -0,0 +1,200 @@
|
||||
package watcherguru
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/xml"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"golang.org/x/net/html"
|
||||
|
||||
"github.com/tradarr/backend/internal/scraper"
|
||||
)
|
||||
|
||||
const baseURL = "https://watcher.guru"
|
||||
|
||||
type WatcherGuru struct {
|
||||
client *http.Client
|
||||
}
|
||||
|
||||
func New() *WatcherGuru {
|
||||
return &WatcherGuru{client: &http.Client{Timeout: 15 * time.Second}}
|
||||
}
|
||||
|
||||
func (w *WatcherGuru) Name() string { return "watcherguru" }
|
||||
|
||||
type rssFeed struct {
|
||||
Channel struct {
|
||||
Items []struct {
|
||||
Title string `xml:"title"`
|
||||
Link string `xml:"link"`
|
||||
PubDate string `xml:"pubDate"`
|
||||
Desc string `xml:"description"`
|
||||
} `xml:"item"`
|
||||
} `xml:"channel"`
|
||||
}
|
||||
|
||||
func (w *WatcherGuru) Scrape(ctx context.Context, _ []string) ([]scraper.Article, error) {
|
||||
// Try RSS feeds first
|
||||
for _, feedURL := range []string{
|
||||
baseURL + "/feed/",
|
||||
baseURL + "/news/feed/",
|
||||
} {
|
||||
articles, err := w.fetchRSS(ctx, feedURL)
|
||||
if err == nil && len(articles) > 0 {
|
||||
fmt.Printf("watcherguru rss: %d articles\n", len(articles))
|
||||
return articles, nil
|
||||
}
|
||||
}
|
||||
|
||||
// Fallback: HTML scraping
|
||||
articles, err := w.scrapeHTML(ctx)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("watcherguru: %w", err)
|
||||
}
|
||||
fmt.Printf("watcherguru html: %d articles\n", len(articles))
|
||||
return articles, nil
|
||||
}
|
||||
|
||||
func (w *WatcherGuru) fetchRSS(ctx context.Context, feedURL string) ([]scraper.Article, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, feedURL, nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("User-Agent", "Mozilla/5.0 (compatible; Tradarr/1.0)")
|
||||
|
||||
resp, err := w.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
return nil, fmt.Errorf("HTTP %d", resp.StatusCode)
|
||||
}
|
||||
|
||||
var feed rssFeed
|
||||
if err := xml.NewDecoder(resp.Body).Decode(&feed); err != nil {
|
||||
return nil, fmt.Errorf("parse RSS: %w", err)
|
||||
}
|
||||
|
||||
var articles []scraper.Article
|
||||
for _, item := range feed.Channel.Items {
|
||||
title := strings.TrimSpace(item.Title)
|
||||
link := strings.TrimSpace(item.Link)
|
||||
if title == "" || link == "" {
|
||||
continue
|
||||
}
|
||||
var publishedAt *time.Time
|
||||
for _, f := range []string{time.RFC1123Z, time.RFC1123, "Mon, 02 Jan 2006 15:04:05 -0700"} {
|
||||
if t, err := time.Parse(f, item.PubDate); err == nil {
|
||||
publishedAt = &t
|
||||
break
|
||||
}
|
||||
}
|
||||
content := strings.TrimSpace(item.Desc)
|
||||
if content == "" {
|
||||
content = title
|
||||
}
|
||||
articles = append(articles, scraper.Article{
|
||||
Title: title,
|
||||
Content: content,
|
||||
URL: link,
|
||||
PublishedAt: publishedAt,
|
||||
})
|
||||
}
|
||||
return articles, nil
|
||||
}
|
||||
|
||||
func (w *WatcherGuru) scrapeHTML(ctx context.Context) ([]scraper.Article, error) {
|
||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, baseURL+"/news/", nil)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
req.Header.Set("User-Agent", "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 Chrome/122.0.0.0 Safari/537.36")
|
||||
req.Header.Set("Accept", "text/html,application/xhtml+xml")
|
||||
|
||||
resp, err := w.client.Do(req)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
|
||||
if resp.StatusCode != http.StatusOK {
|
||||
body, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
|
||||
return nil, fmt.Errorf("HTTP %d: %s", resp.StatusCode, body)
|
||||
}
|
||||
|
||||
doc, err := html.Parse(resp.Body)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("parse HTML: %w", err)
|
||||
}
|
||||
|
||||
var articles []scraper.Article
|
||||
seen := make(map[string]bool)
|
||||
now := time.Now()
|
||||
|
||||
var walk func(*html.Node)
|
||||
walk = func(n *html.Node) {
|
||||
if n.Type == html.ElementNode && (n.Data == "a" || n.Data == "article") {
|
||||
if n.Data == "a" {
|
||||
href := attrVal(n, "href")
|
||||
if href == "" || seen[href] {
|
||||
walk(n.FirstChild)
|
||||
return
|
||||
}
|
||||
// Collect links that look like news articles
|
||||
if strings.Contains(href, "/news/") || strings.Contains(href, "watcher.guru") {
|
||||
text := strings.TrimSpace(nodeText(n))
|
||||
if len(text) > 20 {
|
||||
url := href
|
||||
if !strings.HasPrefix(url, "http") {
|
||||
url = baseURL + url
|
||||
}
|
||||
if !seen[url] {
|
||||
seen[url] = true
|
||||
articles = append(articles, scraper.Article{
|
||||
Title: text,
|
||||
Content: text,
|
||||
URL: url,
|
||||
PublishedAt: &now,
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
for c := n.FirstChild; c != nil; c = c.NextSibling {
|
||||
walk(c)
|
||||
}
|
||||
}
|
||||
walk(doc)
|
||||
|
||||
if len(articles) > 40 {
|
||||
articles = articles[:40]
|
||||
}
|
||||
return articles, nil
|
||||
}
|
||||
|
||||
func attrVal(n *html.Node, key string) string {
|
||||
for _, a := range n.Attr {
|
||||
if a.Key == key {
|
||||
return a.Val
|
||||
}
|
||||
}
|
||||
return ""
|
||||
}
|
||||
|
||||
func nodeText(n *html.Node) string {
|
||||
if n.Type == html.TextNode {
|
||||
return n.Data
|
||||
}
|
||||
var sb strings.Builder
|
||||
for c := n.FirstChild; c != nil; c = c.NextSibling {
|
||||
sb.WriteString(nodeText(c))
|
||||
}
|
||||
return sb.String()
|
||||
}
|
||||
@ -86,8 +86,13 @@ func (y *YahooFinance) fetchSymbol(ctx context.Context, symbol string) ([]scrape
|
||||
return nil, fmt.Errorf("parse RSS: %w", err)
|
||||
}
|
||||
|
||||
const maxPerSymbol = 5
|
||||
|
||||
var articles []scraper.Article
|
||||
for _, item := range feed.Channel.Items {
|
||||
if len(articles) >= maxPerSymbol {
|
||||
break
|
||||
}
|
||||
title := strings.TrimSpace(item.Title)
|
||||
link := strings.TrimSpace(item.Link)
|
||||
if title == "" || link == "" {
|
||||
|
||||
60
docker-compose.prod.yml
Normal file
60
docker-compose.prod.yml
Normal file
@ -0,0 +1,60 @@
|
||||
services:
|
||||
postgres:
|
||||
image: postgres:16-alpine
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
POSTGRES_DB: ${POSTGRES_DB:-tradarr}
|
||||
POSTGRES_USER: ${POSTGRES_USER:-tradarr}
|
||||
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:?POSTGRES_PASSWORD is required}
|
||||
volumes:
|
||||
- pgdata:/var/lib/postgresql/data
|
||||
healthcheck:
|
||||
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-tradarr}"]
|
||||
interval: 10s
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
scraper:
|
||||
image: gitea.anthonybouteiller.ovh/blomios/tradarr-scraper:v1.0.0
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- "3001"
|
||||
|
||||
backend:
|
||||
image: gitea.anthonybouteiller.ovh/blomios/tradarr-backend:v1.0.0
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
scraper:
|
||||
condition: service_started
|
||||
environment:
|
||||
DATABASE_URL: "host=postgres port=5432 user=${POSTGRES_USER:-tradarr} password=${POSTGRES_PASSWORD} dbname=${POSTGRES_DB:-tradarr} sslmode=disable"
|
||||
JWT_SECRET: ${JWT_SECRET:?JWT_SECRET is required}
|
||||
ENCRYPTION_KEY: ${ENCRYPTION_KEY:?ENCRYPTION_KEY must be 32 bytes hex}
|
||||
PORT: "8080"
|
||||
SCRAPER_URL: "http://scraper:3001"
|
||||
ADMIN_EMAIL: ${ADMIN_EMAIL:-admin@tradarr.local}
|
||||
ADMIN_PASSWORD: ${ADMIN_PASSWORD:-changeme}
|
||||
volumes:
|
||||
- /home/anthony/.claude:/root/.claude
|
||||
expose:
|
||||
- "8080"
|
||||
|
||||
ollama:
|
||||
image: ollama/ollama
|
||||
restart: unless-stopped
|
||||
volumes:
|
||||
- ollama_data:/root/.ollama
|
||||
|
||||
frontend:
|
||||
image: gitea.anthonybouteiller.ovh/blomios/tradarr-frontend:v1.0.0
|
||||
restart: unless-stopped
|
||||
depends_on:
|
||||
- backend
|
||||
ports:
|
||||
- "${FRONTEND_PORT:-80}:80"
|
||||
|
||||
volumes:
|
||||
pgdata:
|
||||
ollama_data:
|
||||
@ -14,6 +14,14 @@ services:
|
||||
timeout: 5s
|
||||
retries: 5
|
||||
|
||||
scraper:
|
||||
build:
|
||||
context: ./scraper-service
|
||||
dockerfile: Dockerfile
|
||||
restart: unless-stopped
|
||||
expose:
|
||||
- "3001"
|
||||
|
||||
backend:
|
||||
build:
|
||||
context: ./backend
|
||||
@ -22,13 +30,18 @@ services:
|
||||
depends_on:
|
||||
postgres:
|
||||
condition: service_healthy
|
||||
scraper:
|
||||
condition: service_started
|
||||
environment:
|
||||
DATABASE_URL: "host=postgres port=5432 user=${POSTGRES_USER:-tradarr} password=${POSTGRES_PASSWORD} dbname=${POSTGRES_DB:-tradarr} sslmode=disable"
|
||||
JWT_SECRET: ${JWT_SECRET:?JWT_SECRET is required}
|
||||
ENCRYPTION_KEY: ${ENCRYPTION_KEY:?ENCRYPTION_KEY must be 32 bytes hex}
|
||||
PORT: "8080"
|
||||
SCRAPER_URL: "http://scraper:3001"
|
||||
ADMIN_EMAIL: ${ADMIN_EMAIL:-admin@tradarr.local}
|
||||
ADMIN_PASSWORD: ${ADMIN_PASSWORD:-changeme}
|
||||
volumes:
|
||||
- /home/anthony/.claude:/root/.claude
|
||||
expose:
|
||||
- "8080"
|
||||
|
||||
|
||||
@ -1,10 +1,18 @@
|
||||
FROM node:22-alpine AS builder
|
||||
FROM node:22 AS builder
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends librsvg2-bin && rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
COPY package.json package-lock.json* ./
|
||||
RUN npm ci
|
||||
|
||||
COPY . .
|
||||
|
||||
# Generate PNG icons from SVG source
|
||||
RUN mkdir -p public/icons && \
|
||||
rsvg-convert -w 192 -h 192 public/icon-source.svg -o public/icons/icon-192.png && \
|
||||
rsvg-convert -w 512 -h 512 public/icon-source.svg -o public/icons/icon-512.png
|
||||
|
||||
RUN npm run build
|
||||
|
||||
FROM nginx:alpine
|
||||
|
||||
13
frontend/public/icon-source.svg
Normal file
13
frontend/public/icon-source.svg
Normal file
@ -0,0 +1,13 @@
|
||||
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 512 512">
|
||||
<!-- Background -->
|
||||
<rect width="512" height="512" rx="96" fill="#0f172a"/>
|
||||
<!-- Chart line scaled to center safe zone (~360px) -->
|
||||
<g transform="translate(76, 76) scale(15)">
|
||||
<polyline points="22 7 13.5 15.5 8.5 10.5 2 17"
|
||||
fill="none" stroke="#3b82f6" stroke-width="2.2"
|
||||
stroke-linecap="round" stroke-linejoin="round"/>
|
||||
<polyline points="16 7 22 7 22 13"
|
||||
fill="none" stroke="#3b82f6" stroke-width="2.2"
|
||||
stroke-linecap="round" stroke-linejoin="round"/>
|
||||
</g>
|
||||
</svg>
|
||||
|
After Width: | Height: | Size: 570 B |
@ -4,6 +4,12 @@ export interface AIProvider {
|
||||
id: string; name: string; model: string; endpoint: string
|
||||
is_active: boolean; has_key: boolean
|
||||
}
|
||||
export interface AIRoleConfig { provider_id: string; model: string }
|
||||
export interface AIRoles { summary: AIRoleConfig; report: AIRoleConfig; filter: AIRoleConfig }
|
||||
export interface OllamaModelInfo {
|
||||
name: string; size: number; modified_at: string
|
||||
details: { parameter_size: string; quantization_level: string; family: string }
|
||||
}
|
||||
export interface Source { id: string; name: string; type: string; enabled: boolean }
|
||||
export interface ScrapeJob {
|
||||
id: string; source_id: string; source_name: string; status: string
|
||||
@ -12,6 +18,7 @@ export interface ScrapeJob {
|
||||
articles_found: number; error_msg: string; created_at: string
|
||||
}
|
||||
export interface Setting { key: string; value: string }
|
||||
export interface ScheduleSlot { id?: string; day_of_week: number; hour: number; minute: number }
|
||||
export interface AdminUser { id: string; email: string; role: string; created_at: string }
|
||||
export interface Credential { source_id: string; source_name: string; username: string; has_password: boolean }
|
||||
|
||||
@ -30,6 +37,17 @@ export const adminApi = {
|
||||
activateProvider: (id: string) => api.post<void>(`/admin/ai-providers/${id}/activate`),
|
||||
deleteProvider: (id: string) => api.delete<void>(`/admin/ai-providers/${id}`),
|
||||
listModels: (id: string) => api.get<string[]>(`/admin/ai-providers/${id}/models`),
|
||||
probeModels: (data: { name: string; api_key?: string; endpoint?: string }) =>
|
||||
api.post<string[]>('/admin/ai-providers/probe-models', data),
|
||||
|
||||
// AI Roles
|
||||
getRoles: () => api.get<AIRoles>('/admin/ai-roles'),
|
||||
updateRole: (role: string, data: AIRoleConfig) => api.put<void>(`/admin/ai-roles/${role}`, data),
|
||||
|
||||
// Ollama model management
|
||||
listOllamaModels: () => api.get<OllamaModelInfo[]>('/admin/ollama/models'),
|
||||
pullOllamaModel: (name: string) => api.post<void>('/admin/ollama/pull', { name }),
|
||||
deleteOllamaModel: (name: string) => api.delete<void>(`/admin/ollama/models/${encodeURIComponent(name)}`),
|
||||
|
||||
// Sources
|
||||
listSources: () => api.get<Source[]>('/admin/sources'),
|
||||
@ -44,6 +62,10 @@ export const adminApi = {
|
||||
updateSettings: (settings: Setting[]) => api.put<void>('/admin/settings', { settings }),
|
||||
getDefaultPrompt: () => api.get<{ prompt: string }>('/admin/settings/default-prompt'),
|
||||
|
||||
// Schedule
|
||||
getSchedule: () => api.get<ScheduleSlot[]>('/admin/schedule'),
|
||||
updateSchedule: (slots: ScheduleSlot[]) => api.put<void>('/admin/schedule', { slots }),
|
||||
|
||||
// Users
|
||||
listUsers: () => api.get<AdminUser[]>('/admin/users'),
|
||||
updateUser: (id: string, email: string, role: string) =>
|
||||
|
||||
@ -14,6 +14,12 @@ async function request<T>(path: string, options: RequestInit = {}): Promise<T> {
|
||||
...options.headers,
|
||||
},
|
||||
})
|
||||
if (res.status === 401) {
|
||||
localStorage.removeItem('token')
|
||||
localStorage.removeItem('user')
|
||||
window.location.href = '/login'
|
||||
throw new Error('Session expirée')
|
||||
}
|
||||
if (!res.ok) {
|
||||
const err = await res.json().catch(() => ({ error: res.statusText }))
|
||||
throw new Error(err.error || res.statusText)
|
||||
|
||||
20
frontend/src/api/reports.ts
Normal file
20
frontend/src/api/reports.ts
Normal file
@ -0,0 +1,20 @@
|
||||
import { api } from './client'
|
||||
|
||||
export interface Report {
|
||||
id: string
|
||||
user_id: string
|
||||
summary_id: string | null
|
||||
context_excerpt: string
|
||||
question: string
|
||||
answer: string
|
||||
status: 'generating' | 'done' | 'error'
|
||||
error_msg: string
|
||||
created_at: string
|
||||
}
|
||||
|
||||
export const reportsApi = {
|
||||
list: () => api.get<Report[]>('/reports'),
|
||||
create: (data: { summary_id?: string; excerpts: string[]; question: string }) =>
|
||||
api.post<Report>('/reports', data),
|
||||
delete: (id: string) => api.delete<void>(`/reports/${id}`),
|
||||
}
|
||||
@ -11,4 +11,5 @@ export interface Summary {
|
||||
export const summariesApi = {
|
||||
list: (limit = 10) => api.get<Summary[]>(`/summaries?limit=${limit}`),
|
||||
generate: () => api.post<Summary>('/summaries/generate'),
|
||||
status: () => api.get<{ generating: boolean }>('/summaries/status'),
|
||||
}
|
||||
|
||||
@ -1,11 +1,12 @@
|
||||
import { NavLink } from 'react-router-dom'
|
||||
import { LayoutDashboard, Newspaper, Star, Settings } from 'lucide-react'
|
||||
import { LayoutDashboard, Newspaper, Star, Settings, FileText } from 'lucide-react'
|
||||
import { cn } from '@/lib/cn'
|
||||
|
||||
const items = [
|
||||
{ to: '/', icon: LayoutDashboard, label: 'Dashboard' },
|
||||
{ to: '/feed', icon: Newspaper, label: 'Actus' },
|
||||
{ to: '/watchlist', icon: Star, label: 'Watchlist' },
|
||||
{ to: '/reports', icon: FileText, label: 'Rapports' },
|
||||
{ to: '/admin', icon: Settings, label: 'Admin' },
|
||||
]
|
||||
|
||||
|
||||
@ -1,5 +1,5 @@
|
||||
import { NavLink } from 'react-router-dom'
|
||||
import { LayoutDashboard, Newspaper, Star, Settings, Key, Cpu, Database, ClipboardList, Users, LogOut, TrendingUp } from 'lucide-react'
|
||||
import { LayoutDashboard, Newspaper, Star, Settings, Key, Cpu, Database, ClipboardList, Users, LogOut, TrendingUp, CalendarDays, FileText } from 'lucide-react'
|
||||
import { useAuth } from '@/lib/auth'
|
||||
import { cn } from '@/lib/cn'
|
||||
|
||||
@ -7,6 +7,7 @@ const navItems = [
|
||||
{ to: '/', icon: LayoutDashboard, label: 'Dashboard' },
|
||||
{ to: '/feed', icon: Newspaper, label: 'Actualités' },
|
||||
{ to: '/watchlist', icon: Star, label: 'Watchlist' },
|
||||
{ to: '/reports', icon: FileText, label: 'Rapports' },
|
||||
]
|
||||
|
||||
const adminItems = [
|
||||
@ -15,6 +16,7 @@ const adminItems = [
|
||||
{ to: '/admin/sources', icon: Database, label: 'Sources' },
|
||||
{ to: '/admin/jobs', icon: ClipboardList, label: 'Jobs' },
|
||||
{ to: '/admin/users', icon: Users, label: 'Utilisateurs' },
|
||||
{ to: '/admin/schedule', icon: CalendarDays, label: 'Planning' },
|
||||
{ to: '/admin/settings', icon: Settings, label: 'Paramètres' },
|
||||
]
|
||||
|
||||
|
||||
28
frontend/src/components/ui/markdown.tsx
Normal file
28
frontend/src/components/ui/markdown.tsx
Normal file
@ -0,0 +1,28 @@
|
||||
import React from 'react'
|
||||
|
||||
function renderInline(text: string): React.ReactNode {
|
||||
const parts = text.split(/(\*\*[^*]+\*\*)/g)
|
||||
return parts.map((part, i) =>
|
||||
part.startsWith('**') && part.endsWith('**')
|
||||
? <strong key={i}>{part.slice(2, -2)}</strong>
|
||||
: part
|
||||
)
|
||||
}
|
||||
|
||||
export function Markdown({ content, className }: { content: string; className?: string }) {
|
||||
const lines = content.split('\n')
|
||||
return (
|
||||
<div className={`space-y-1 text-sm leading-relaxed select-text ${className ?? ''}`}>
|
||||
{lines.map((line, i) => {
|
||||
if (line.startsWith('##### ')) return <h5 key={i} className="text-xs font-semibold uppercase tracking-wide text-muted-foreground mt-3">{renderInline(line.slice(6))}</h5>
|
||||
if (line.startsWith('#### ')) return <h4 key={i} className="text-sm font-semibold mt-3">{renderInline(line.slice(5))}</h4>
|
||||
if (line.startsWith('### ')) return <h3 key={i} className="text-sm font-semibold text-primary mt-4">{renderInline(line.slice(4))}</h3>
|
||||
if (line.startsWith('## ')) return <h2 key={i} className="text-base font-bold mt-5 first:mt-0 border-b pb-1">{renderInline(line.slice(3))}</h2>
|
||||
if (line.startsWith('# ')) return <h1 key={i} className="text-lg font-bold mt-5 first:mt-0">{renderInline(line.slice(2))}</h1>
|
||||
if (line.startsWith('- ') || line.startsWith('* ')) return <li key={i} className="ml-4 text-muted-foreground list-disc">{renderInline(line.slice(2))}</li>
|
||||
if (line.trim() === '') return <div key={i} className="h-2" />
|
||||
return <p key={i} className="text-muted-foreground">{renderInline(line)}</p>
|
||||
})}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@ -37,3 +37,8 @@
|
||||
::-webkit-scrollbar { width: 6px; height: 6px; }
|
||||
::-webkit-scrollbar-track { @apply bg-muted; }
|
||||
::-webkit-scrollbar-thumb { @apply bg-border rounded-full; }
|
||||
|
||||
@layer utilities {
|
||||
.scrollbar-none { scrollbar-width: none; }
|
||||
.scrollbar-none::-webkit-scrollbar { display: none; }
|
||||
}
|
||||
|
||||
@ -11,6 +11,8 @@ import { Sources } from '@/pages/admin/Sources'
|
||||
import { Jobs } from '@/pages/admin/Jobs'
|
||||
import { AdminUsers } from '@/pages/admin/AdminUsers'
|
||||
import { AdminSettings } from '@/pages/admin/AdminSettings'
|
||||
import { Schedule } from '@/pages/admin/Schedule'
|
||||
import { Reports } from '@/pages/Reports'
|
||||
|
||||
export const router = createBrowserRouter([
|
||||
{ path: '/login', element: <Login /> },
|
||||
@ -20,6 +22,7 @@ export const router = createBrowserRouter([
|
||||
{ path: '/', element: <Dashboard /> },
|
||||
{ path: '/feed', element: <Feed /> },
|
||||
{ path: '/watchlist', element: <Watchlist /> },
|
||||
{ path: '/reports', element: <Reports /> },
|
||||
{
|
||||
path: '/admin',
|
||||
element: <AdminLayout />,
|
||||
@ -31,6 +34,7 @@ export const router = createBrowserRouter([
|
||||
{ path: 'jobs', element: <Jobs /> },
|
||||
{ path: 'users', element: <AdminUsers /> },
|
||||
{ path: 'settings', element: <AdminSettings /> },
|
||||
{ path: 'schedule', element: <Schedule /> },
|
||||
],
|
||||
},
|
||||
],
|
||||
|
||||
@ -1,29 +1,152 @@
|
||||
import { useState, useEffect } from 'react'
|
||||
import { TrendingUp, Clock, Sparkles } from 'lucide-react'
|
||||
import { useState, useEffect, useRef, useCallback } from 'react'
|
||||
import { TrendingUp, Clock, Sparkles, MessageSquarePlus, Loader2, Plus, X, Send, ChevronDown, ChevronUp } from 'lucide-react'
|
||||
import { summariesApi, type Summary } from '@/api/summaries'
|
||||
import { reportsApi } from '@/api/reports'
|
||||
import { assetsApi, type Asset } from '@/api/assets'
|
||||
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
|
||||
import { Markdown } from '@/components/ui/markdown'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Badge } from '@/components/ui/badge'
|
||||
import { Spinner } from '@/components/ui/spinner'
|
||||
import { useAuth } from '@/lib/auth'
|
||||
|
||||
function SummaryContent({ content }: { content: string }) {
|
||||
const lines = content.split('\n')
|
||||
// ── Text-selection floating button ─────────────────────────────────────────
|
||||
|
||||
function useTextSelection(containerRef: React.RefObject<HTMLElement>) {
|
||||
const [selection, setSelection] = useState<{ text: string; x: number; y: number } | null>(null)
|
||||
|
||||
useEffect(() => {
|
||||
function readSelection() {
|
||||
const sel = window.getSelection()
|
||||
const text = sel?.toString().trim()
|
||||
if (!text || !containerRef.current) { setSelection(null); return }
|
||||
const range = sel!.getRangeAt(0)
|
||||
const rect = range.getBoundingClientRect()
|
||||
const containerRect = containerRef.current.getBoundingClientRect()
|
||||
setSelection({
|
||||
text,
|
||||
x: rect.left - containerRect.left + rect.width / 2,
|
||||
y: rect.top - containerRect.top - 8,
|
||||
})
|
||||
}
|
||||
function onMouseUp() { readSelection() }
|
||||
// On mobile, selectionchange fires after the user lifts their finger
|
||||
function onSelectionChange() {
|
||||
const sel = window.getSelection()
|
||||
if (!sel?.toString().trim()) setSelection(null)
|
||||
else readSelection()
|
||||
}
|
||||
function onPointerDown(e: PointerEvent) {
|
||||
if (!(e.target as Element).closest('[data-context-action]')) setSelection(null)
|
||||
}
|
||||
document.addEventListener('mouseup', onMouseUp)
|
||||
document.addEventListener('selectionchange', onSelectionChange)
|
||||
document.addEventListener('pointerdown', onPointerDown)
|
||||
return () => {
|
||||
document.removeEventListener('mouseup', onMouseUp)
|
||||
document.removeEventListener('selectionchange', onSelectionChange)
|
||||
document.removeEventListener('pointerdown', onPointerDown)
|
||||
}
|
||||
}, [containerRef])
|
||||
|
||||
return selection
|
||||
}
|
||||
|
||||
// ── Context panel (extraits + question) ────────────────────────────────────
|
||||
|
||||
function ContextPanel({
|
||||
excerpts,
|
||||
onRemove,
|
||||
onClear,
|
||||
onSubmit,
|
||||
}: {
|
||||
excerpts: string[]
|
||||
onRemove: (i: number) => void
|
||||
onClear: () => void
|
||||
onSubmit: (question: string) => Promise<void>
|
||||
}) {
|
||||
const [question, setQuestion] = useState('')
|
||||
const [submitting, setSubmitting] = useState(false)
|
||||
const [submitted, setSubmitted] = useState(false)
|
||||
const [showExcerpts, setShowExcerpts] = useState(false)
|
||||
|
||||
async function submit() {
|
||||
if (!question.trim() || submitting) return
|
||||
setSubmitting(true)
|
||||
try {
|
||||
await onSubmit(question)
|
||||
setSubmitted(true)
|
||||
setQuestion('')
|
||||
setTimeout(() => setSubmitted(false), 2000)
|
||||
} finally {
|
||||
setSubmitting(false)
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-2 text-sm leading-relaxed">
|
||||
{lines.map((line, i) => {
|
||||
if (line.startsWith('## ')) return <h2 key={i} className="text-base font-semibold mt-4 first:mt-0">{line.slice(3)}</h2>
|
||||
if (line.startsWith('### ')) return <h3 key={i} className="font-medium mt-3">{line.slice(4)}</h3>
|
||||
if (line.startsWith('- ')) return <li key={i} className="ml-4 text-muted-foreground">{line.slice(2)}</li>
|
||||
if (line.startsWith('**') && line.endsWith('**')) return <p key={i} className="font-semibold">{line.slice(2, -2)}</p>
|
||||
if (line.trim() === '') return <div key={i} className="h-1" />
|
||||
return <p key={i} className="text-muted-foreground">{line}</p>
|
||||
})}
|
||||
<div className="fixed bottom-16 left-0 right-0 z-40 flex flex-col bg-card border-t shadow-2xl overflow-hidden md:bottom-4 md:left-auto md:right-4 md:w-96 md:rounded-xl md:border">
|
||||
{/* Header */}
|
||||
<div className="flex items-center justify-between px-4 py-3 border-b bg-primary/5">
|
||||
<button
|
||||
className="text-sm font-semibold flex items-center gap-2 hover:text-primary transition-colors"
|
||||
onClick={() => setShowExcerpts(v => !v)}
|
||||
>
|
||||
<MessageSquarePlus className="h-4 w-4 text-primary" />
|
||||
Contexte ({excerpts.length} extrait{excerpts.length > 1 ? 's' : ''})
|
||||
<span className="md:hidden">{showExcerpts ? <ChevronDown className="h-3 w-3" /> : <ChevronUp className="h-3 w-3" />}</span>
|
||||
</button>
|
||||
<button onClick={onClear} className="text-xs text-muted-foreground hover:text-destructive transition-colors">
|
||||
Tout effacer
|
||||
</button>
|
||||
</div>
|
||||
|
||||
{/* Extraits — toujours visible sur desktop, togglable sur mobile */}
|
||||
<div className={`flex-1 overflow-y-auto px-4 py-3 space-y-2 min-h-0 max-h-[25vh] ${showExcerpts ? 'block' : 'hidden'} md:block`}>
|
||||
{excerpts.map((e, i) => (
|
||||
<div key={i} className="flex items-start gap-2 group">
|
||||
<div className="flex-1 rounded bg-muted/60 px-2 py-1.5 text-xs text-muted-foreground italic line-clamp-3">
|
||||
« {e} »
|
||||
</div>
|
||||
<button
|
||||
onClick={() => onRemove(i)}
|
||||
className="mt-1 shrink-0 text-muted-foreground hover:text-destructive md:opacity-0 md:group-hover:opacity-100 transition-opacity"
|
||||
>
|
||||
<X className="h-3 w-3" />
|
||||
</button>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
|
||||
{/* Question */}
|
||||
<div className="border-t px-4 py-3 space-y-2">
|
||||
<textarea
|
||||
className="w-full rounded border border-input bg-background px-3 py-2 text-sm focus:outline-none focus:ring-1 focus:ring-ring resize-none"
|
||||
rows={2}
|
||||
placeholder="Votre question…"
|
||||
value={question}
|
||||
onChange={e => setQuestion(e.target.value)}
|
||||
onKeyDown={e => { if (e.key === 'Enter' && (e.ctrlKey || e.metaKey)) submit() }}
|
||||
/>
|
||||
<Button size="sm" className="w-full" onClick={submit} disabled={!question.trim() || submitting || submitted}>
|
||||
{submitting
|
||||
? <><Loader2 className="h-3 w-3 animate-spin" /> Envoi…</>
|
||||
: submitted
|
||||
? 'Envoyé !'
|
||||
: <><Send className="h-3 w-3" /> Envoyer (Ctrl+Entrée)</>}
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ── Summary content renderer ────────────────────────────────────────────────
|
||||
|
||||
function SummaryContent({ content }: { content: string }) {
|
||||
return <Markdown content={content} />
|
||||
}
|
||||
|
||||
// ── Dashboard ───────────────────────────────────────────────────────────────
|
||||
|
||||
export function Dashboard() {
|
||||
const { user } = useAuth()
|
||||
const [summaries, setSummaries] = useState<Summary[]>([])
|
||||
@ -31,9 +154,26 @@ export function Dashboard() {
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [generating, setGenerating] = useState(false)
|
||||
const [current, setCurrent] = useState<Summary | null>(null)
|
||||
const [excerpts, setExcerpts] = useState<string[]>([])
|
||||
|
||||
const summaryRef = useRef<HTMLDivElement>(null)
|
||||
const textSel = useTextSelection(summaryRef as React.RefObject<HTMLElement>)
|
||||
|
||||
useEffect(() => { load() }, [])
|
||||
|
||||
// Poll generating status every 5s
|
||||
useEffect(() => {
|
||||
const interval = setInterval(async () => {
|
||||
try {
|
||||
const status = await summariesApi.status()
|
||||
const wasGenerating = generating
|
||||
setGenerating(status.generating)
|
||||
if (wasGenerating && !status.generating) load()
|
||||
} catch { /* ignore */ }
|
||||
}, 5000)
|
||||
return () => clearInterval(interval)
|
||||
}, [generating])
|
||||
|
||||
async function load() {
|
||||
setLoading(true)
|
||||
try {
|
||||
@ -55,8 +195,31 @@ export function Dashboard() {
|
||||
} finally { setGenerating(false) }
|
||||
}
|
||||
|
||||
const addExcerpt = useCallback(() => {
|
||||
if (!textSel) return
|
||||
setExcerpts(prev => [...prev, textSel.text])
|
||||
window.getSelection()?.removeAllRanges()
|
||||
}, [textSel])
|
||||
|
||||
async function submitQuestion(question: string) {
|
||||
await reportsApi.create({
|
||||
summary_id: current?.id,
|
||||
excerpts,
|
||||
question,
|
||||
})
|
||||
setExcerpts([])
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="p-4 md:p-6 space-y-6">
|
||||
{/* Generating banner */}
|
||||
{generating && (
|
||||
<div className="flex items-center gap-3 rounded-md bg-primary/10 border border-primary/20 px-4 py-3 text-sm text-primary">
|
||||
<Loader2 className="h-4 w-4 animate-spin shrink-0" />
|
||||
Génération d'un résumé IA en cours — cela peut prendre plusieurs minutes…
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Header */}
|
||||
<div className="flex flex-wrap items-start justify-between gap-4">
|
||||
<div>
|
||||
@ -105,7 +268,37 @@ export function Dashboard() {
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent>
|
||||
<div ref={summaryRef} className="relative">
|
||||
<SummaryContent content={current.content} />
|
||||
{/* Desktop: floating button above selection */}
|
||||
{textSel && (
|
||||
<button
|
||||
data-context-action
|
||||
onClick={addExcerpt}
|
||||
className="absolute z-10 hidden md:flex items-center gap-1 rounded-full bg-primary text-primary-foreground text-xs px-2 py-1 shadow-lg hover:bg-primary/90 transition-colors"
|
||||
style={{ left: textSel.x, top: textSel.y, transform: 'translate(-50%, -100%)' }}
|
||||
>
|
||||
<Plus className="h-3 w-3" />
|
||||
Ajouter au contexte
|
||||
</button>
|
||||
)}
|
||||
</div>
|
||||
{/* Mobile: fixed bottom bar — avoids conflict with native selection menu */}
|
||||
{textSel && (
|
||||
<div className="md:hidden fixed bottom-16 left-0 right-0 z-50 flex justify-center px-4 pointer-events-none">
|
||||
<button
|
||||
data-context-action
|
||||
onClick={addExcerpt}
|
||||
className="pointer-events-auto flex items-center gap-2 rounded-full bg-primary text-primary-foreground text-sm px-4 py-2.5 shadow-xl"
|
||||
>
|
||||
<Plus className="h-4 w-4" />
|
||||
Ajouter au contexte
|
||||
</button>
|
||||
</div>
|
||||
)}
|
||||
<p className="text-xs text-muted-foreground mt-4 italic">
|
||||
Sélectionnez du texte pour l'ajouter au contexte, puis posez une question dans le panneau qui apparaît.
|
||||
</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
) : (
|
||||
@ -141,6 +334,16 @@ export function Dashboard() {
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
|
||||
{/* Panneau contexte flottant */}
|
||||
{excerpts.length > 0 && (
|
||||
<ContextPanel
|
||||
excerpts={excerpts}
|
||||
onRemove={i => setExcerpts(prev => prev.filter((_, idx) => idx !== i))}
|
||||
onClear={() => setExcerpts([])}
|
||||
onSubmit={submitQuestion}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
128
frontend/src/pages/Reports.tsx
Normal file
128
frontend/src/pages/Reports.tsx
Normal file
@ -0,0 +1,128 @@
|
||||
import { useState, useEffect, useCallback } from 'react'
|
||||
import { Trash2, FileText, Clock, Loader2, XCircle, AlertCircle } from 'lucide-react'
|
||||
import { reportsApi, type Report } from '@/api/reports'
|
||||
import { Card, CardContent } from '@/components/ui/card'
|
||||
import { Markdown } from '@/components/ui/markdown'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Spinner } from '@/components/ui/spinner'
|
||||
import { Badge } from '@/components/ui/badge'
|
||||
|
||||
function StatusBadge({ status }: { status: Report['status'] }) {
|
||||
if (status === 'generating') return (
|
||||
<Badge variant="secondary" className="gap-1 text-xs">
|
||||
<Loader2 className="h-3 w-3 animate-spin" /> En cours
|
||||
</Badge>
|
||||
)
|
||||
if (status === 'error') return (
|
||||
<Badge variant="destructive" className="gap-1 text-xs">
|
||||
<AlertCircle className="h-3 w-3" /> Erreur
|
||||
</Badge>
|
||||
)
|
||||
return null
|
||||
}
|
||||
|
||||
export function Reports() {
|
||||
const [reports, setReports] = useState<Report[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
|
||||
const load = useCallback(async () => {
|
||||
try { setReports((await reportsApi.list()) ?? []) }
|
||||
finally { setLoading(false) }
|
||||
}, [])
|
||||
|
||||
useEffect(() => { load() }, [load])
|
||||
|
||||
// Poll toutes les 3s tant qu'il y a des rapports en cours
|
||||
useEffect(() => {
|
||||
const hasGenerating = reports.some(r => r.status === 'generating')
|
||||
if (!hasGenerating) return
|
||||
const interval = setInterval(load, 3000)
|
||||
return () => clearInterval(interval)
|
||||
}, [reports, load])
|
||||
|
||||
async function remove(id: string) {
|
||||
await reportsApi.delete(id)
|
||||
setReports(prev => prev.filter(r => r.id !== id))
|
||||
}
|
||||
|
||||
if (loading) return <div className="flex justify-center py-20"><Spinner /></div>
|
||||
|
||||
return (
|
||||
<div className="p-4 md:p-6 space-y-6">
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold">Rapports IA</h1>
|
||||
<p className="text-muted-foreground text-sm">
|
||||
Vos questions posées sur des extraits de résumés, avec les réponses de l'IA.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
{reports.length === 0 ? (
|
||||
<Card className="border-dashed">
|
||||
<CardContent className="py-12 text-center text-muted-foreground">
|
||||
<FileText className="h-8 w-8 mx-auto mb-3 opacity-50" />
|
||||
<p>Aucun rapport enregistré</p>
|
||||
<p className="text-xs mt-1">Sélectionnez du texte dans un résumé et posez une question à l'IA.</p>
|
||||
</CardContent>
|
||||
</Card>
|
||||
) : (
|
||||
<div className="space-y-4">
|
||||
{reports.map(r => (
|
||||
<Card key={r.id} className={r.status === 'generating' ? 'border-primary/30' : ''}>
|
||||
<CardContent className="pt-4 pb-4 space-y-3">
|
||||
{/* Meta */}
|
||||
<div className="flex items-start justify-between gap-4">
|
||||
<div className="flex items-center gap-2 flex-wrap">
|
||||
<div className="flex items-center gap-1 text-xs text-muted-foreground">
|
||||
<Clock className="h-3 w-3" />
|
||||
{new Date(r.created_at).toLocaleString('fr-FR')}
|
||||
</div>
|
||||
<StatusBadge status={r.status} />
|
||||
</div>
|
||||
<Button
|
||||
variant="ghost"
|
||||
size="sm"
|
||||
className="h-6 w-6 p-0 text-muted-foreground hover:text-destructive shrink-0"
|
||||
onClick={() => remove(r.id)}
|
||||
title={r.status === 'generating' ? 'Annuler' : 'Supprimer'}
|
||||
>
|
||||
{r.status === 'generating'
|
||||
? <XCircle className="h-3 w-3" />
|
||||
: <Trash2 className="h-3 w-3" />}
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{/* Extraits de contexte */}
|
||||
{r.context_excerpt.split('\n\n---\n\n').map((excerpt, i) => (
|
||||
<div key={i} className="rounded bg-muted/60 px-3 py-2 text-xs text-muted-foreground italic">
|
||||
« {excerpt} »
|
||||
</div>
|
||||
))}
|
||||
|
||||
{/* Question */}
|
||||
<p className="text-sm font-medium">{r.question}</p>
|
||||
|
||||
{/* Réponse */}
|
||||
{r.status === 'generating' && (
|
||||
<div className="flex items-center gap-2 text-sm text-muted-foreground">
|
||||
<Loader2 className="h-4 w-4 animate-spin text-primary" />
|
||||
L'IA génère la réponse…
|
||||
</div>
|
||||
)}
|
||||
{r.status === 'done' && (
|
||||
<div className="border-t pt-3">
|
||||
<Markdown content={r.answer} />
|
||||
</div>
|
||||
)}
|
||||
{r.status === 'error' && (
|
||||
<div className="text-sm text-destructive border-t pt-3">
|
||||
Erreur : {r.error_msg || 'Une erreur est survenue lors de la génération.'}
|
||||
</div>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@ -1,6 +1,6 @@
|
||||
import { useState, useEffect } from 'react'
|
||||
import { Plus, Trash2, CheckCircle, RefreshCw } from 'lucide-react'
|
||||
import { adminApi, type AIProvider } from '@/api/admin'
|
||||
import { useState, useEffect, useCallback } from 'react'
|
||||
import { Plus, Trash2, Star, Download, HardDrive, Cpu, ChevronDown, ChevronUp, X } from 'lucide-react'
|
||||
import { adminApi, type AIProvider, type AIRoles, type OllamaModelInfo } from '@/api/admin'
|
||||
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Input } from '@/components/ui/input'
|
||||
@ -9,184 +9,644 @@ import { Select } from '@/components/ui/select'
|
||||
import { Badge } from '@/components/ui/badge'
|
||||
import { Spinner } from '@/components/ui/spinner'
|
||||
|
||||
const PROVIDER_NAMES = ['openai', 'anthropic', 'gemini', 'ollama'] as const
|
||||
const PROVIDER_NAMES = ['openai', 'anthropic', 'gemini', 'ollama', 'claudecode'] as const
|
||||
|
||||
export function AIProviders() {
|
||||
const [providers, setProviders] = useState<AIProvider[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [showForm, setShowForm] = useState(false)
|
||||
const [form, setForm] = useState({ name: 'openai', api_key: '', model: '', endpoint: '' })
|
||||
const isOllama = form.name === 'ollama'
|
||||
const [models, setModels] = useState<Record<string, string[]>>({})
|
||||
const [loadingModels, setLoadingModels] = useState<string | null>(null)
|
||||
const [editId, setEditId] = useState<string | null>(null)
|
||||
const [saving, setSaving] = useState(false)
|
||||
const [error, setError] = useState('')
|
||||
const CUSTOM_MODELS_KEY = 'ollama_custom_models'
|
||||
|
||||
useEffect(() => { load() }, [])
|
||||
const KNOWN_OLLAMA_MODELS: { name: string; family: string; tags: { tag: string; size: string }[] }[] = [
|
||||
{ name: 'qwen3', family: 'Qwen3', tags: [{ tag: '0.6b', size: '~0.4 GB' }, { tag: '1.7b', size: '~1.1 GB' }, { tag: '4b', size: '~2.6 GB' }, { tag: '8b', size: '~5.2 GB' }, { tag: '14b', size: '~9.3 GB' }, { tag: '32b', size: '~20 GB' }] },
|
||||
{ name: 'qwen2.5', family: 'Qwen2.5', tags: [{ tag: '0.5b', size: '~0.4 GB' }, { tag: '1.5b', size: '~1 GB' }, { tag: '3b', size: '~2 GB' }, { tag: '7b', size: '~4.7 GB' }, { tag: '14b', size: '~9 GB' }, { tag: '32b', size: '~20 GB' }] },
|
||||
{ name: 'llama3.2', family: 'Llama 3.2', tags: [{ tag: '1b', size: '~1.3 GB' }, { tag: '3b', size: '~2 GB' }] },
|
||||
{ name: 'llama3.1', family: 'Llama 3.1', tags: [{ tag: '8b', size: '~4.9 GB' }, { tag: '70b', size: '~40 GB' }] },
|
||||
{ name: 'gemma3', family: 'Gemma 3', tags: [{ tag: '1b', size: '~0.8 GB' }, { tag: '4b', size: '~3.3 GB' }, { tag: '12b', size: '~8 GB' }, { tag: '27b', size: '~17 GB' }] },
|
||||
{ name: 'mistral', family: 'Mistral', tags: [{ tag: '7b', size: '~4.1 GB' }] },
|
||||
{ name: 'phi4', family: 'Phi-4', tags: [{ tag: 'latest', size: '~9 GB' }] },
|
||||
{ name: 'phi4-mini', family: 'Phi-4 Mini', tags: [{ tag: 'latest', size: '~2.5 GB' }] },
|
||||
{ name: 'deepseek-r1', family: 'DeepSeek-R1', tags: [{ tag: '1.5b', size: '~1.1 GB' }, { tag: '7b', size: '~4.7 GB' }, { tag: '8b', size: '~4.9 GB' }, { tag: '14b', size: '~9 GB' }, { tag: '32b', size: '~20 GB' }] },
|
||||
]
|
||||
|
||||
async function load() {
|
||||
setLoading(true)
|
||||
try { setProviders((await adminApi.listProviders()) ?? []) } finally { setLoading(false) }
|
||||
function formatSize(bytes: number): string {
|
||||
if (bytes >= 1e9) return `${(bytes / 1e9).toFixed(1)} GB`
|
||||
if (bytes >= 1e6) return `${(bytes / 1e6).toFixed(0)} MB`
|
||||
return `${bytes} B`
|
||||
}
|
||||
|
||||
async function loadModels(id: string) {
|
||||
setLoadingModels(id)
|
||||
function loadCustomModels(): string[] {
|
||||
try { return JSON.parse(localStorage.getItem(CUSTOM_MODELS_KEY) ?? '[]') } catch { return [] }
|
||||
}
|
||||
function saveCustomModels(models: string[]) {
|
||||
localStorage.setItem(CUSTOM_MODELS_KEY, JSON.stringify(models))
|
||||
}
|
||||
|
||||
// ── Confirmation dialog ────────────────────────────────────────────────────
|
||||
|
||||
function ConfirmDownloadDialog({ modelName, onConfirm, onCancel }: {
|
||||
modelName: string
|
||||
onConfirm: () => void
|
||||
onCancel: () => void
|
||||
}) {
|
||||
return (
|
||||
<div className="fixed inset-0 z-50 flex items-center justify-center bg-black/50">
|
||||
<div className="bg-card border rounded-xl shadow-2xl p-6 w-full max-w-sm mx-4 space-y-4">
|
||||
<div className="flex items-start justify-between">
|
||||
<div>
|
||||
<p className="font-semibold">Télécharger ce modèle ?</p>
|
||||
<p className="text-sm text-muted-foreground mt-1">
|
||||
<span className="font-mono text-foreground">{modelName}</span> sera téléchargé depuis Ollama Hub.
|
||||
Le téléchargement peut prendre plusieurs minutes.
|
||||
</p>
|
||||
</div>
|
||||
<button onClick={onCancel} className="text-muted-foreground hover:text-foreground ml-3 shrink-0">
|
||||
<X className="h-4 w-4" />
|
||||
</button>
|
||||
</div>
|
||||
<div className="flex gap-2 justify-end">
|
||||
<Button variant="outline" size="sm" onClick={onCancel}>Annuler</Button>
|
||||
<Button size="sm" onClick={onConfirm}>
|
||||
<Download className="h-3 w-3 mr-1" /> Télécharger
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ── Shared model catalogue ─────────────────────────────────────────────────
|
||||
|
||||
function ModelCatalogue({
|
||||
installedNames,
|
||||
pulling,
|
||||
onPull,
|
||||
}: {
|
||||
installedNames: Set<string>
|
||||
pulling: string | null
|
||||
onPull: (name: string) => void
|
||||
}) {
|
||||
const [customName, setCustomName] = useState('')
|
||||
const [pendingModel, setPendingModel] = useState<string | null>(null)
|
||||
const [customModels, setCustomModels] = useState<string[]>(loadCustomModels)
|
||||
|
||||
// All model names in the hardcoded catalogue
|
||||
const knownNames = new Set(
|
||||
KNOWN_OLLAMA_MODELS.flatMap(f => f.tags.map(t => `${f.name}:${t.tag}`))
|
||||
)
|
||||
|
||||
function requestPull(name: string) {
|
||||
setPendingModel(name)
|
||||
}
|
||||
|
||||
function confirmPull() {
|
||||
if (!pendingModel) return
|
||||
// If not in hardcoded catalogue, add to custom list
|
||||
if (!knownNames.has(pendingModel)) {
|
||||
const updated = [...new Set([...customModels, pendingModel])]
|
||||
setCustomModels(updated)
|
||||
saveCustomModels(updated)
|
||||
}
|
||||
onPull(pendingModel)
|
||||
setPendingModel(null)
|
||||
}
|
||||
|
||||
function submitCustom() {
|
||||
const name = customName.trim()
|
||||
if (!name) return
|
||||
const fullName = name.includes(':') ? name : `${name}:latest`
|
||||
setCustomName('')
|
||||
requestPull(fullName)
|
||||
}
|
||||
|
||||
// Families to display: hardcoded + custom entries
|
||||
const customEntries = customModels.filter(m => !knownNames.has(m))
|
||||
|
||||
return (
|
||||
<>
|
||||
{pendingModel && (
|
||||
<ConfirmDownloadDialog
|
||||
modelName={pendingModel}
|
||||
onConfirm={confirmPull}
|
||||
onCancel={() => setPendingModel(null)}
|
||||
/>
|
||||
)}
|
||||
|
||||
<div className="space-y-4">
|
||||
{/* Free-text input */}
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground mb-1.5">Nom personnalisé</p>
|
||||
<div className="flex gap-2">
|
||||
<Input
|
||||
className="h-8 text-sm font-mono"
|
||||
placeholder="ex: llama3.2:3b, mymodel:latest…"
|
||||
value={customName}
|
||||
onChange={e => setCustomName(e.target.value)}
|
||||
onKeyDown={e => e.key === 'Enter' && submitCustom()}
|
||||
disabled={pulling !== null}
|
||||
/>
|
||||
<Button size="sm" className="h-8 shrink-0" onClick={submitCustom}
|
||||
disabled={!customName.trim() || pulling !== null}>
|
||||
<Download className="h-3 w-3" /> Installer
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
{/* Custom models section */}
|
||||
{customEntries.length > 0 && (
|
||||
<div>
|
||||
<p className="text-xs font-semibold text-muted-foreground mb-1.5">Personnalisés</p>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{customEntries.map(fullName => {
|
||||
const installed = installedNames.has(fullName)
|
||||
const isPulling = pulling === fullName
|
||||
return <ModelTag key={fullName} fullName={fullName} size="?" installed={installed} isPulling={isPulling} pulling={pulling} onRequest={requestPull} />
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{/* Hardcoded catalogue */}
|
||||
<div className="space-y-3">
|
||||
{KNOWN_OLLAMA_MODELS.map(family => (
|
||||
<div key={family.name}>
|
||||
<p className="text-xs font-semibold text-muted-foreground mb-1.5">{family.family}</p>
|
||||
<div className="flex flex-wrap gap-1.5">
|
||||
{family.tags.map(({ tag, size }) => {
|
||||
const fullName = `${family.name}:${tag}`
|
||||
const installed = installedNames.has(fullName)
|
||||
const isPulling = pulling === fullName
|
||||
return <ModelTag key={fullName} fullName={fullName} size={size} installed={installed} isPulling={isPulling} pulling={pulling} onRequest={requestPull} />
|
||||
})}
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
</div>
|
||||
</>
|
||||
)
|
||||
}
|
||||
|
||||
function ModelTag({ fullName, size, installed, isPulling, pulling, onRequest }: {
|
||||
fullName: string; size: string
|
||||
installed: boolean; isPulling: boolean; pulling: string | null
|
||||
onRequest: (name: string) => void
|
||||
}) {
|
||||
return (
|
||||
<button
|
||||
type="button"
|
||||
disabled={installed || pulling !== null}
|
||||
onClick={() => !installed && !isPulling && onRequest(fullName)}
|
||||
className={[
|
||||
'flex items-center gap-1.5 rounded-md border px-2 py-1 text-xs transition-colors',
|
||||
installed
|
||||
? 'border-primary/40 bg-primary/5 text-primary cursor-default'
|
||||
: isPulling
|
||||
? 'border-border bg-muted text-muted-foreground cursor-wait'
|
||||
: pulling !== null
|
||||
? 'border-border bg-background text-muted-foreground opacity-50 cursor-not-allowed'
|
||||
: 'border-border bg-background hover:border-primary/60 hover:bg-primary/5 cursor-pointer',
|
||||
].join(' ')}
|
||||
>
|
||||
{isPulling
|
||||
? <><Spinner className="h-2.5 w-2.5" /> Téléchargement…</>
|
||||
: installed
|
||||
? <><span className="text-[10px]">✓</span> {fullName}</>
|
||||
: <><Download className="h-2.5 w-2.5" /> {fullName} {size !== '?' && <span className="text-muted-foreground">{size}</span>}</>
|
||||
}
|
||||
</button>
|
||||
)
|
||||
}
|
||||
|
||||
// ── Cloud model picker (OpenAI / Anthropic / Gemini / …) ──────────────────
|
||||
|
||||
function CloudModelPicker({ value, providerName, apiKey, endpoint, onChange }: {
|
||||
value: string; providerName: string; apiKey: string; endpoint: string
|
||||
onChange: (model: string) => void
|
||||
}) {
|
||||
const [models, setModels] = useState<string[]>([])
|
||||
const [loading, setLoading] = useState(false)
|
||||
const [loadError, setLoadError] = useState('')
|
||||
const noKeyNeeded = providerName === 'claudecode'
|
||||
|
||||
useEffect(() => {
|
||||
if (noKeyNeeded) loadModels()
|
||||
// eslint-disable-next-line react-hooks/exhaustive-deps
|
||||
}, [providerName])
|
||||
|
||||
async function loadModels() {
|
||||
if (!noKeyNeeded && !apiKey) { setLoadError('Renseigne la clé API d\'abord'); return }
|
||||
setLoading(true); setLoadError('')
|
||||
try {
|
||||
const m = await adminApi.listModels(id)
|
||||
setModels(prev => ({ ...prev, [id]: m }))
|
||||
} catch { /* silently ignore */ } finally { setLoadingModels(null) }
|
||||
}
|
||||
|
||||
async function save() {
|
||||
setSaving(true); setError('')
|
||||
try {
|
||||
if (editId) {
|
||||
await adminApi.updateProvider(editId, form)
|
||||
} else {
|
||||
await adminApi.createProvider(form)
|
||||
}
|
||||
setShowForm(false); setEditId(null)
|
||||
setForm({ name: 'openai', api_key: '', model: '', endpoint: '' })
|
||||
await load()
|
||||
} catch (e) { setError(e instanceof Error ? e.message : 'Erreur') } finally { setSaving(false) }
|
||||
}
|
||||
|
||||
async function activate(id: string) {
|
||||
await adminApi.activateProvider(id)
|
||||
await load()
|
||||
}
|
||||
|
||||
async function remove(id: string) {
|
||||
if (!confirm('Supprimer ce fournisseur ?')) return
|
||||
await adminApi.deleteProvider(id)
|
||||
await load()
|
||||
}
|
||||
|
||||
function startEdit(p: AIProvider) {
|
||||
setEditId(p.id)
|
||||
setForm({ name: p.name, api_key: '', model: p.model, endpoint: p.endpoint })
|
||||
setShowForm(true)
|
||||
const list = await adminApi.probeModels({ name: providerName, api_key: apiKey, endpoint })
|
||||
setModels(list ?? [])
|
||||
if ((list ?? []).length === 0) setLoadError('Aucun modèle trouvé')
|
||||
} catch (e) {
|
||||
setLoadError(e instanceof Error ? e.message : 'Erreur')
|
||||
} finally { setLoading(false) }
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold">Fournisseurs IA</h1>
|
||||
<p className="text-muted-foreground text-sm">Configurez les fournisseurs IA et sélectionnez le modèle actif</p>
|
||||
</div>
|
||||
<Button onClick={() => { setShowForm(true); setEditId(null) }}>
|
||||
<Plus className="h-4 w-4" /> Ajouter
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
{showForm && (
|
||||
<Card>
|
||||
<CardHeader><CardTitle>{editId ? 'Modifier' : 'Nouveau fournisseur'}</CardTitle></CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<div className="grid gap-4 sm:grid-cols-2">
|
||||
<div className="space-y-1">
|
||||
<Label>Fournisseur</Label>
|
||||
<Select
|
||||
value={form.name}
|
||||
onChange={e => {
|
||||
const name = e.target.value
|
||||
setForm(f => ({
|
||||
...f,
|
||||
name,
|
||||
endpoint: name === 'ollama' ? 'http://ollama:11434' : '',
|
||||
api_key: '',
|
||||
}))
|
||||
}}
|
||||
disabled={!!editId}
|
||||
>
|
||||
{PROVIDER_NAMES.map(n => <option key={n} value={n}>{n}</option>)}
|
||||
</Select>
|
||||
</div>
|
||||
{!isOllama && (
|
||||
<div className="space-y-1">
|
||||
<Label>Clé API {editId && <span className="text-muted-foreground">(laisser vide pour conserver)</span>}</Label>
|
||||
<Input type="password" placeholder="sk-..." value={form.api_key} onChange={e => setForm(f => ({ ...f, api_key: e.target.value }))} />
|
||||
</div>
|
||||
)}
|
||||
<div className="space-y-1">
|
||||
<Label>Modèle</Label>
|
||||
<Input placeholder="gpt-4o-mini, claude-sonnet-4-6…" value={form.model} onChange={e => setForm(f => ({ ...f, model: e.target.value }))} />
|
||||
</div>
|
||||
{isOllama && (
|
||||
<div className="space-y-1">
|
||||
<Label>Endpoint Ollama</Label>
|
||||
<Input value="http://ollama:11434" readOnly className="opacity-60 cursor-not-allowed" />
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
{error && <p className="text-sm text-destructive">{error}</p>}
|
||||
<div className="space-y-1.5">
|
||||
<div className="flex gap-2">
|
||||
<Button onClick={save} disabled={saving}>{saving ? <Spinner className="h-4 w-4" /> : 'Enregistrer'}</Button>
|
||||
<Button variant="outline" onClick={() => { setShowForm(false); setEditId(null) }}>Annuler</Button>
|
||||
<Input
|
||||
placeholder="gpt-4o-mini, claude-sonnet-4-6…"
|
||||
value={value}
|
||||
onChange={e => onChange(e.target.value)}
|
||||
list="cloud-models-list"
|
||||
className="flex-1"
|
||||
/>
|
||||
<Button type="button" variant="outline" size="sm" className="shrink-0" onClick={loadModels} disabled={loading}>
|
||||
{loading ? <Spinner className="h-3 w-3" /> : 'Charger'}
|
||||
</Button>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
{models.length > 0 && (
|
||||
<datalist id="cloud-models-list">
|
||||
{models.map(m => <option key={m} value={m} />)}
|
||||
</datalist>
|
||||
)}
|
||||
|
||||
{loading ? (
|
||||
<div className="flex justify-center py-12"><Spinner /></div>
|
||||
) : (
|
||||
<div className="space-y-3">
|
||||
{providers.map(p => (
|
||||
<Card key={p.id} className={p.is_active ? 'border-primary/50' : ''}>
|
||||
<CardContent className="flex flex-wrap items-center gap-4 py-4">
|
||||
<div className="flex-1 min-w-0">
|
||||
<div className="flex items-center gap-2 flex-wrap">
|
||||
<span className="font-semibold capitalize">{p.name}</span>
|
||||
{p.is_active && <Badge variant="default">Actif</Badge>}
|
||||
{!p.has_key && p.name !== 'ollama' && <Badge variant="outline" className="text-yellow-500 border-yellow-500">Sans clé</Badge>}
|
||||
</div>
|
||||
<div className="text-sm text-muted-foreground mt-1 flex gap-4 flex-wrap">
|
||||
{p.model && <span>Modèle : <strong>{p.model}</strong></span>}
|
||||
{p.endpoint && <span>Endpoint : {p.endpoint}</span>}
|
||||
</div>
|
||||
{/* Dropdown modèles disponibles */}
|
||||
{models[p.id] && models[p.id].length > 0 && (
|
||||
<div className="mt-2 space-y-1">
|
||||
<Label className="text-xs">Choisir un modèle :</Label>
|
||||
<Select
|
||||
className="w-full max-w-xs"
|
||||
value={p.model}
|
||||
onChange={async e => {
|
||||
await adminApi.updateProvider(p.id, { name: p.name, model: e.target.value, endpoint: p.endpoint })
|
||||
await load()
|
||||
}}
|
||||
>
|
||||
{models[p.id].map(m => <option key={m} value={m}>{m}</option>)}
|
||||
{models.length > 0 && (
|
||||
<Select value={value} onChange={e => onChange(e.target.value)}>
|
||||
<option value="">— Sélectionner un modèle —</option>
|
||||
{models.map(m => <option key={m} value={m}>{m}</option>)}
|
||||
</Select>
|
||||
</div>
|
||||
)}
|
||||
{loadError && <p className="text-xs text-destructive">{loadError}</p>}
|
||||
</div>
|
||||
<div className="flex items-center gap-2 flex-wrap">
|
||||
<Button variant="outline" size="sm" onClick={() => loadModels(p.id)} disabled={loadingModels === p.id}>
|
||||
{loadingModels === p.id ? <Spinner className="h-3 w-3" /> : <RefreshCw className="h-3 w-3" />}
|
||||
Modèles
|
||||
</Button>
|
||||
<Button variant="outline" size="sm" onClick={() => startEdit(p)}>Modifier</Button>
|
||||
{!p.is_active && (
|
||||
<Button size="sm" onClick={() => activate(p.id)}>
|
||||
<CheckCircle className="h-3 w-3" /> Activer
|
||||
</Button>
|
||||
)}
|
||||
<Button variant="destructive" size="sm" onClick={() => remove(p.id)}>
|
||||
<Trash2 className="h-3 w-3" />
|
||||
</Button>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
)
|
||||
}
|
||||
|
||||
// ── Ollama model picker (inside provider form) ─────────────────────────────
|
||||
|
||||
function OllamaModelPicker({ value, installedModels, onSelect, onRefresh }: {
|
||||
value: string
|
||||
installedModels: OllamaModelInfo[]
|
||||
onSelect: (name: string) => void
|
||||
onRefresh: () => Promise<OllamaModelInfo[]>
|
||||
}) {
|
||||
const [showCatalogue, setShowCatalogue] = useState(installedModels.length === 0)
|
||||
const [pulling, setPulling] = useState<string | null>(null)
|
||||
const installedNames = new Set(installedModels.map(m => m.name))
|
||||
|
||||
async function pull(fullName: string) {
|
||||
setPulling(fullName)
|
||||
try {
|
||||
await adminApi.pullOllamaModel(fullName)
|
||||
await new Promise<void>(resolve => {
|
||||
const iv = setInterval(async () => {
|
||||
try {
|
||||
const updated = await onRefresh()
|
||||
if (updated.some(m => m.name === fullName)) { clearInterval(iv); onSelect(fullName); setShowCatalogue(false); resolve() }
|
||||
} catch { /* ignore */ }
|
||||
}, 4000)
|
||||
setTimeout(() => { clearInterval(iv); resolve() }, 1800000)
|
||||
})
|
||||
} finally { setPulling(null) }
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-2">
|
||||
<Select value={value} onChange={e => onSelect(e.target.value)}>
|
||||
<option value="">— Choisir un modèle installé —</option>
|
||||
{installedModels.map(m => (
|
||||
<option key={m.name} value={m.name}>
|
||||
{m.name}{m.details.parameter_size ? ` (${m.details.parameter_size})` : ''}
|
||||
</option>
|
||||
))}
|
||||
{providers.length === 0 && (
|
||||
<Card><CardContent className="py-8 text-center text-muted-foreground">Aucun fournisseur configuré</CardContent></Card>
|
||||
)}
|
||||
</Select>
|
||||
<button type="button" className="flex items-center gap-1 text-xs text-primary hover:underline"
|
||||
onClick={() => setShowCatalogue(v => !v)}>
|
||||
{showCatalogue ? <ChevronUp className="h-3 w-3" /> : <ChevronDown className="h-3 w-3" />}
|
||||
{showCatalogue ? 'Masquer le catalogue' : 'Installer un nouveau modèle'}
|
||||
</button>
|
||||
{showCatalogue && (
|
||||
<div className="rounded-lg border bg-muted/30 p-3 max-h-72 overflow-y-auto">
|
||||
<ModelCatalogue installedNames={installedNames} pulling={pulling} onPull={pull} />
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ── Role Assignment ────────────────────────────────────────────────────────
|
||||
|
||||
const ROLE_LABELS: Record<string, { label: string; desc: string }> = {
|
||||
summary: { label: 'Résumés', desc: 'Génération du résumé quotidien (passe 2)' },
|
||||
report: { label: 'Rapports', desc: 'Réponses aux questions sur les résumés' },
|
||||
filter: { label: 'Filtre articles', desc: 'Sélection des articles pertinents (passe 1)' },
|
||||
}
|
||||
|
||||
function RoleCard({ role, providers, currentProviderID, onSave }: {
|
||||
role: string; providers: AIProvider[]; currentProviderID: string
|
||||
onSave: (providerID: string) => Promise<void>
|
||||
}) {
|
||||
const [providerID, setProviderID] = useState(currentProviderID)
|
||||
const [saving, setSaving] = useState(false)
|
||||
const [saved, setSaved] = useState(false)
|
||||
useEffect(() => { setProviderID(currentProviderID) }, [currentProviderID])
|
||||
const { label, desc } = ROLE_LABELS[role]
|
||||
async function save() {
|
||||
setSaving(true)
|
||||
try { await onSave(providerID); setSaved(true); setTimeout(() => setSaved(false), 2000) }
|
||||
finally { setSaving(false) }
|
||||
}
|
||||
return (
|
||||
<div className="grid gap-3 sm:grid-cols-[1fr_auto] items-end">
|
||||
<div>
|
||||
<Label className="text-xs font-semibold uppercase tracking-wide text-muted-foreground">{label}</Label>
|
||||
<p className="text-xs text-muted-foreground mb-2">{desc}</p>
|
||||
<Select value={providerID} onChange={e => setProviderID(e.target.value)}>
|
||||
<option value="">— Fournisseur par défaut —</option>
|
||||
{providers.map(p => <option key={p.id} value={p.id}>{p.name}{p.model ? ` — ${p.model}` : ''}</option>)}
|
||||
</Select>
|
||||
</div>
|
||||
<Button size="sm" onClick={save} disabled={saving} className="whitespace-nowrap">
|
||||
{saving ? <Spinner className="h-3 w-3" /> : saved ? '✓ Sauvegardé' : 'Appliquer'}
|
||||
</Button>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
// ── Ollama model manager card ──────────────────────────────────────────────
|
||||
|
||||
function OllamaModelsCard({ installedModels, onRefresh, onDelete }: {
|
||||
installedModels: OllamaModelInfo[]
|
||||
onRefresh: () => Promise<OllamaModelInfo[]>
|
||||
onDelete: (name: string) => Promise<void>
|
||||
}) {
|
||||
const [showCatalogue, setShowCatalogue] = useState(false)
|
||||
const [pulling, setPulling] = useState<string | null>(null)
|
||||
const [deleting, setDeleting] = useState<string | null>(null)
|
||||
const installedNames = new Set(installedModels.map(m => m.name))
|
||||
|
||||
async function pull(fullName: string) {
|
||||
setPulling(fullName)
|
||||
try {
|
||||
await adminApi.pullOllamaModel(fullName)
|
||||
await new Promise<void>(resolve => {
|
||||
const iv = setInterval(async () => {
|
||||
try {
|
||||
const updated = await onRefresh()
|
||||
if (updated.some(m => m.name === fullName)) { clearInterval(iv); resolve() }
|
||||
} catch { /* ignore */ }
|
||||
}, 4000)
|
||||
setTimeout(() => { clearInterval(iv); resolve() }, 1800000)
|
||||
})
|
||||
} finally { setPulling(null) }
|
||||
}
|
||||
|
||||
async function remove(name: string) {
|
||||
setDeleting(name)
|
||||
try { await onDelete(name) } finally { setDeleting(null) }
|
||||
}
|
||||
|
||||
return (
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<div className="flex items-center justify-between">
|
||||
<CardTitle className="flex items-center gap-2 text-base">
|
||||
<HardDrive className="h-4 w-4" /> Modèles Ollama
|
||||
</CardTitle>
|
||||
<Button variant="outline" size="sm" onClick={() => setShowCatalogue(v => !v)}>
|
||||
{showCatalogue ? <><ChevronUp className="h-3.5 w-3.5" /> Masquer</> : <><Download className="h-3.5 w-3.5" /> Installer</>}
|
||||
</Button>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<div>
|
||||
<h3 className="text-sm font-semibold mb-2 flex items-center gap-1.5">
|
||||
<Cpu className="h-3.5 w-3.5" /> Installés ({installedModels.length})
|
||||
</h3>
|
||||
{installedModels.length === 0 ? (
|
||||
<p className="text-sm text-muted-foreground italic">Aucun modèle installé</p>
|
||||
) : (
|
||||
<div className="space-y-1.5">
|
||||
{installedModels.map(m => (
|
||||
<div key={m.name} className="flex items-center justify-between rounded-md border bg-muted/30 px-3 py-2">
|
||||
<div>
|
||||
<span className="font-mono text-sm font-medium">{m.name}</span>
|
||||
<span className="ml-3 text-xs text-muted-foreground">
|
||||
{formatSize(m.size)}
|
||||
{m.details.parameter_size && <> · {m.details.parameter_size}</>}
|
||||
{m.details.quantization_level && <> · {m.details.quantization_level}</>}
|
||||
</span>
|
||||
</div>
|
||||
<Button variant="ghost" size="sm"
|
||||
className="text-destructive hover:text-destructive h-7 w-7 p-0"
|
||||
onClick={() => remove(m.name)} disabled={deleting === m.name}>
|
||||
{deleting === m.name ? <Spinner className="h-3 w-3" /> : <Trash2 className="h-3 w-3" />}
|
||||
</Button>
|
||||
</div>
|
||||
))}
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
{showCatalogue && (
|
||||
<div className="rounded-lg border bg-muted/30 p-4">
|
||||
<p className="text-sm font-semibold mb-3">Catalogue</p>
|
||||
<ModelCatalogue installedNames={installedNames} pulling={pulling} onPull={pull} />
|
||||
</div>
|
||||
)}
|
||||
</CardContent>
|
||||
</Card>
|
||||
)
|
||||
}
|
||||
|
||||
// ── Main Page ──────────────────────────────────────────────────────────────
|
||||
|
||||
export function AIProviders() {
|
||||
const [providers, setProviders] = useState<AIProvider[]>([])
|
||||
const [roles, setRoles] = useState<AIRoles | null>(null)
|
||||
const [ollamaModels, setOllamaModels] = useState<OllamaModelInfo[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [showForm, setShowForm] = useState(false)
|
||||
const [form, setForm] = useState({ name: 'openai', api_key: '', model: '', endpoint: '' })
|
||||
const [editId, setEditId] = useState<string | null>(null)
|
||||
const [saving, setSaving] = useState(false)
|
||||
const [error, setError] = useState('')
|
||||
|
||||
const isOllamaForm = form.name === 'ollama'
|
||||
const isClaudeCodeForm = form.name === 'claudecode'
|
||||
|
||||
const loadOllamaModels = useCallback(async (): Promise<OllamaModelInfo[]> => {
|
||||
try {
|
||||
console.log('[ollama] fetching installed models…')
|
||||
const m = await adminApi.listOllamaModels()
|
||||
const list = m ?? []
|
||||
console.log(`[ollama] ${list.length} model(s):`, list.map(x => x.name))
|
||||
setOllamaModels(list)
|
||||
return list
|
||||
} catch (e) {
|
||||
console.error('[ollama] error:', e)
|
||||
return []
|
||||
}
|
||||
}, [])
|
||||
|
||||
const load = useCallback(async () => {
|
||||
setLoading(true)
|
||||
try {
|
||||
const [p, r] = await Promise.all([adminApi.listProviders(), adminApi.getRoles()])
|
||||
setProviders(p ?? [])
|
||||
setRoles(r)
|
||||
await loadOllamaModels()
|
||||
} finally { setLoading(false) }
|
||||
}, [loadOllamaModels])
|
||||
|
||||
useEffect(() => { load() }, [load])
|
||||
|
||||
async function save() {
|
||||
setSaving(true); setError('')
|
||||
try {
|
||||
if (editId) { await adminApi.updateProvider(editId, form) }
|
||||
else { await adminApi.createProvider(form) }
|
||||
setShowForm(false); setEditId(null)
|
||||
setForm({ name: 'openai', api_key: '', model: '', endpoint: '' })
|
||||
await load()
|
||||
} catch (e) { setError(e instanceof Error ? e.message : 'Erreur') }
|
||||
finally { setSaving(false) }
|
||||
}
|
||||
|
||||
async function setDefault(id: string) { await adminApi.activateProvider(id); await load() }
|
||||
async function remove(id: string) {
|
||||
if (!confirm('Supprimer ce fournisseur ?')) return
|
||||
await adminApi.deleteProvider(id); await load()
|
||||
}
|
||||
function startEdit(p: AIProvider) {
|
||||
setEditId(p.id); setForm({ name: p.name, api_key: '', model: p.model, endpoint: p.endpoint }); setShowForm(true)
|
||||
}
|
||||
async function saveRole(role: string, providerID: string) {
|
||||
await adminApi.updateRole(role, { provider_id: providerID, model: '' }); await load()
|
||||
}
|
||||
async function deleteOllamaModel(name: string) {
|
||||
await adminApi.deleteOllamaModel(name); await loadOllamaModels()
|
||||
}
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold">Fournisseurs IA</h1>
|
||||
<p className="text-muted-foreground text-sm">Configurez les fournisseurs et assignez un modèle à chaque tâche</p>
|
||||
</div>
|
||||
|
||||
{loading ? (
|
||||
<div className="flex justify-center py-12"><Spinner /></div>
|
||||
) : (
|
||||
<>
|
||||
{/* Role assignments */}
|
||||
{roles && providers.length > 0 && (
|
||||
<Card>
|
||||
<CardHeader><CardTitle className="text-base">Assignation des tâches</CardTitle></CardHeader>
|
||||
<CardContent className="space-y-5 divide-y">
|
||||
{(['summary', 'report', 'filter'] as const).map(role => (
|
||||
<div key={role} className={role !== 'summary' ? 'pt-5' : ''}>
|
||||
<RoleCard role={role} providers={providers}
|
||||
currentProviderID={roles[role].provider_id}
|
||||
onSave={pid => saveRole(role, pid)} />
|
||||
</div>
|
||||
))}
|
||||
</CardContent>
|
||||
</Card>
|
||||
)}
|
||||
|
||||
{/* Providers list */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<div className="flex items-center justify-between">
|
||||
<CardTitle className="text-base">Fournisseurs configurés</CardTitle>
|
||||
<Button size="sm" onClick={() => { setShowForm(true); setEditId(null); setForm({ name: 'openai', api_key: '', model: '', endpoint: '' }) }}>
|
||||
<Plus className="h-4 w-4" /> Ajouter
|
||||
</Button>
|
||||
</div>
|
||||
</CardHeader>
|
||||
<CardContent className="space-y-3">
|
||||
{showForm && (
|
||||
<div className="rounded-lg border p-4 space-y-4 bg-muted/30">
|
||||
<p className="font-medium text-sm">{editId ? 'Modifier le fournisseur' : 'Nouveau fournisseur'}</p>
|
||||
<div className="grid gap-3 sm:grid-cols-2">
|
||||
<div className="space-y-1">
|
||||
<Label>Fournisseur</Label>
|
||||
<Select value={form.name} disabled={!!editId}
|
||||
onChange={e => {
|
||||
const name = e.target.value
|
||||
setForm(f => ({ ...f, name, model: '', api_key: '', endpoint: name === 'ollama' ? 'http://ollama:11434' : '' }))
|
||||
}}>
|
||||
{PROVIDER_NAMES.map(n => <option key={n} value={n}>{n}</option>)}
|
||||
</Select>
|
||||
</div>
|
||||
{!isOllamaForm && !isClaudeCodeForm && (
|
||||
<div className="space-y-1">
|
||||
<Label>Clé API {editId && <span className="text-muted-foreground text-xs">(vide = conserver)</span>}</Label>
|
||||
<Input type="password" placeholder="sk-…" value={form.api_key} onChange={e => setForm(f => ({ ...f, api_key: e.target.value }))} />
|
||||
</div>
|
||||
)}
|
||||
{isOllamaForm && (
|
||||
<div className="space-y-1">
|
||||
<Label>Endpoint</Label>
|
||||
<Input value={form.endpoint || 'http://ollama:11434'} onChange={e => setForm(f => ({ ...f, endpoint: e.target.value }))} />
|
||||
</div>
|
||||
)}
|
||||
</div>
|
||||
<div className="space-y-1">
|
||||
<Label>Modèle par défaut</Label>
|
||||
{isOllamaForm ? (
|
||||
<OllamaModelPicker value={form.model} installedModels={ollamaModels}
|
||||
onSelect={m => setForm(f => ({ ...f, model: m }))}
|
||||
onRefresh={loadOllamaModels} />
|
||||
) : (
|
||||
<CloudModelPicker
|
||||
value={form.model}
|
||||
providerName={form.name}
|
||||
apiKey={form.api_key}
|
||||
endpoint={form.endpoint}
|
||||
onChange={m => setForm(f => ({ ...f, model: m }))}
|
||||
/>
|
||||
)}
|
||||
</div>
|
||||
{error && <p className="text-sm text-destructive">{error}</p>}
|
||||
<div className="flex gap-2">
|
||||
<Button size="sm" onClick={save} disabled={saving}>{saving ? <Spinner className="h-4 w-4" /> : 'Enregistrer'}</Button>
|
||||
<Button size="sm" variant="outline" onClick={() => { setShowForm(false); setEditId(null) }}>Annuler</Button>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
|
||||
{providers.length === 0 && !showForm && (
|
||||
<p className="text-sm text-muted-foreground text-center py-4">Aucun fournisseur configuré</p>
|
||||
)}
|
||||
|
||||
{providers.map(p => (
|
||||
<div key={p.id} className={`flex flex-wrap items-center gap-3 rounded-md border px-3 py-2.5 ${p.is_active ? 'border-primary/50 bg-primary/5' : ''}`}>
|
||||
<div className="flex-1 min-w-0">
|
||||
<div className="flex items-center gap-2 flex-wrap">
|
||||
<span className="font-semibold capitalize text-sm">{p.name}</span>
|
||||
{p.is_active && <Badge variant="default" className="text-xs">Défaut</Badge>}
|
||||
{!p.has_key && p.name !== 'ollama' && p.name !== 'claudecode' && <Badge variant="outline" className="text-yellow-500 border-yellow-500 text-xs">Sans clé</Badge>}
|
||||
</div>
|
||||
<div className="text-xs text-muted-foreground flex gap-3 mt-0.5">
|
||||
{p.model && <span>Modèle : <strong>{p.model}</strong></span>}
|
||||
{p.endpoint && <span>{p.endpoint}</span>}
|
||||
</div>
|
||||
</div>
|
||||
<div className="flex items-center gap-1.5">
|
||||
<Button variant="ghost" size="sm" className="h-7 px-2 text-xs" onClick={() => startEdit(p)}>Modifier</Button>
|
||||
{!p.is_active && (
|
||||
<Button size="sm" variant="outline" className="h-7 px-2 text-xs" onClick={() => setDefault(p.id)}>
|
||||
<Star className="h-3 w-3 mr-1" /> Défaut
|
||||
</Button>
|
||||
)}
|
||||
<Button variant="ghost" size="sm" className="h-7 w-7 p-0 text-destructive hover:text-destructive" onClick={() => remove(p.id)}>
|
||||
<Trash2 className="h-3 w-3" />
|
||||
</Button>
|
||||
</div>
|
||||
</div>
|
||||
))}
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Ollama model manager — always shown so user can install even without provider configured */}
|
||||
<OllamaModelsCard
|
||||
installedModels={ollamaModels}
|
||||
onRefresh={loadOllamaModels}
|
||||
onDelete={deleteOllamaModel}
|
||||
/>
|
||||
</>
|
||||
)}
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@ -1,12 +1,47 @@
|
||||
import { Outlet, Navigate } from 'react-router-dom'
|
||||
import { NavLink, Outlet, Navigate } from 'react-router-dom'
|
||||
import { Cpu, Key, Database, ClipboardList, Users, CalendarDays, Settings } from 'lucide-react'
|
||||
import { useAuth } from '@/lib/auth'
|
||||
import { cn } from '@/lib/cn'
|
||||
|
||||
const adminItems = [
|
||||
{ to: '/admin/ai', icon: Cpu, label: 'Fournisseurs IA' },
|
||||
{ to: '/admin/credentials', icon: Key, label: 'Identifiants' },
|
||||
{ to: '/admin/sources', icon: Database, label: 'Sources' },
|
||||
{ to: '/admin/jobs', icon: ClipboardList,label: 'Jobs' },
|
||||
{ to: '/admin/users', icon: Users, label: 'Utilisateurs' },
|
||||
{ to: '/admin/schedule', icon: CalendarDays, label: 'Planning' },
|
||||
{ to: '/admin/settings', icon: Settings, label: 'Paramètres' },
|
||||
]
|
||||
|
||||
export function AdminLayout() {
|
||||
const { isAdmin } = useAuth()
|
||||
if (!isAdmin) return <Navigate to="/" replace />
|
||||
return (
|
||||
<div>
|
||||
{/* Mobile sub-nav — scrollable horizontal tabs */}
|
||||
<nav className="md:hidden flex overflow-x-auto border-b bg-card scrollbar-none sticky top-0 z-10">
|
||||
{adminItems.map(({ to, icon: Icon, label }) => (
|
||||
<NavLink
|
||||
key={to}
|
||||
to={to}
|
||||
className={({ isActive }) =>
|
||||
cn(
|
||||
'flex shrink-0 flex-col items-center gap-1 px-4 py-3 text-xs font-medium transition-colors border-b-2',
|
||||
isActive
|
||||
? 'border-primary text-primary'
|
||||
: 'border-transparent text-muted-foreground'
|
||||
)
|
||||
}
|
||||
>
|
||||
<Icon className="h-4 w-4" />
|
||||
{label}
|
||||
</NavLink>
|
||||
))}
|
||||
</nav>
|
||||
|
||||
<div className="p-6">
|
||||
<Outlet />
|
||||
</div>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
|
||||
@ -8,19 +8,27 @@ import { Label } from '@/components/ui/label'
|
||||
import { Spinner } from '@/components/ui/spinner'
|
||||
|
||||
const NUMERIC_SETTINGS: Record<string, { label: string; description: string }> = {
|
||||
scrape_interval_minutes: { label: 'Intervalle de scraping (minutes)', description: 'Fréquence de récupération des actualités' },
|
||||
articles_lookback_hours: { label: 'Fenêtre d\'analyse (heures)', description: 'Période couverte pour les résumés IA' },
|
||||
summary_max_articles: { label: 'Articles max par résumé', description: 'Nombre maximum d\'articles envoyés à l\'IA' },
|
||||
summary_max_articles: { label: 'Articles max par résumé', description: 'Nombre maximum d\'articles envoyés à l\'IA pour la passe 2 (résumé)' },
|
||||
filter_batch_size: { label: 'Taille des batches (filtre)', description: 'Nombre d\'articles par appel IA lors de la passe 1 (filtrage). Réduire pour des réponses plus rapides.' },
|
||||
}
|
||||
|
||||
const COMMON_TIMEZONES = [
|
||||
'Europe/Paris', 'Europe/London', 'Europe/Berlin', 'Europe/Madrid', 'Europe/Rome',
|
||||
'America/New_York', 'America/Chicago', 'America/Los_Angeles',
|
||||
'Asia/Tokyo', 'Asia/Hong_Kong', 'Asia/Singapore',
|
||||
'UTC',
|
||||
]
|
||||
|
||||
export function AdminSettings() {
|
||||
const [settings, setSettings] = useState<Setting[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [values, setValues] = useState<Record<string, string>>({})
|
||||
const [defaultPrompt, setDefaultPrompt] = useState('')
|
||||
const [saving, setSaving] = useState(false)
|
||||
const [savingTz, setSavingTz] = useState(false)
|
||||
const [savingPrompt, setSavingPrompt] = useState(false)
|
||||
const [saved, setSaved] = useState(false)
|
||||
const [savedTz, setSavedTz] = useState(false)
|
||||
const [savedPrompt, setSavedPrompt] = useState(false)
|
||||
|
||||
useEffect(() => { load() }, [])
|
||||
@ -53,6 +61,13 @@ export function AdminSettings() {
|
||||
setTimeout(() => setSaved(false), 2000)
|
||||
}
|
||||
|
||||
async function saveTimezone() {
|
||||
setSavingTz(true); setSavedTz(false)
|
||||
await adminApi.updateSettings([{ key: 'timezone', value: values['timezone'] ?? 'Europe/Paris' }])
|
||||
setSavingTz(false); setSavedTz(true)
|
||||
setTimeout(() => setSavedTz(false), 2000)
|
||||
}
|
||||
|
||||
async function savePrompt() {
|
||||
setSavingPrompt(true); setSavedPrompt(false)
|
||||
await adminApi.updateSettings([{ key: 'ai_system_prompt', value: values['ai_system_prompt'] ?? '' }])
|
||||
@ -75,6 +90,35 @@ export function AdminSettings() {
|
||||
<p className="text-muted-foreground text-sm">Configuration globale du service</p>
|
||||
</div>
|
||||
|
||||
{/* Fuseau horaire */}
|
||||
<Card>
|
||||
<CardHeader><CardTitle>Fuseau horaire</CardTitle></CardHeader>
|
||||
<CardContent className="space-y-4">
|
||||
<div className="space-y-1">
|
||||
<Label>Timezone (IANA)</Label>
|
||||
<p className="text-xs text-muted-foreground">
|
||||
Utilisé pour le planning de scraping et l'horodatage des résumés
|
||||
</p>
|
||||
<div className="flex gap-2 max-w-sm">
|
||||
<Input
|
||||
list="tz-list"
|
||||
value={values['timezone'] ?? 'Europe/Paris'}
|
||||
onChange={e => setValues(v => ({ ...v, timezone: e.target.value }))}
|
||||
placeholder="Europe/Paris"
|
||||
/>
|
||||
<datalist id="tz-list">
|
||||
{COMMON_TIMEZONES.map(tz => <option key={tz} value={tz} />)}
|
||||
</datalist>
|
||||
</div>
|
||||
</div>
|
||||
<Button onClick={saveTimezone} disabled={savingTz}>
|
||||
{savingTz ? <Spinner className="h-4 w-4" /> : <Save className="h-4 w-4" />}
|
||||
{savedTz ? 'Enregistré !' : 'Enregistrer'}
|
||||
</Button>
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Paramètres généraux */}
|
||||
<Card>
|
||||
<CardHeader><CardTitle>Paramètres généraux</CardTitle></CardHeader>
|
||||
<CardContent className="space-y-6">
|
||||
@ -101,6 +145,7 @@ export function AdminSettings() {
|
||||
</CardContent>
|
||||
</Card>
|
||||
|
||||
{/* Contexte IA */}
|
||||
<Card>
|
||||
<CardHeader>
|
||||
<CardTitle>Contexte IA</CardTitle>
|
||||
|
||||
145
frontend/src/pages/admin/Schedule.tsx
Normal file
145
frontend/src/pages/admin/Schedule.tsx
Normal file
@ -0,0 +1,145 @@
|
||||
import { useState, useEffect } from 'react'
|
||||
import { Plus, Trash2, Save } from 'lucide-react'
|
||||
import { adminApi, type ScheduleSlot } from '@/api/admin'
|
||||
import { Card, CardContent, CardHeader, CardTitle } from '@/components/ui/card'
|
||||
import { Button } from '@/components/ui/button'
|
||||
import { Spinner } from '@/components/ui/spinner'
|
||||
|
||||
const DAYS = [
|
||||
{ label: 'Lundi', short: 'LUN', value: 1 },
|
||||
{ label: 'Mardi', short: 'MAR', value: 2 },
|
||||
{ label: 'Mercredi', short: 'MER', value: 3 },
|
||||
{ label: 'Jeudi', short: 'JEU', value: 4 },
|
||||
{ label: 'Vendredi', short: 'VEN', value: 5 },
|
||||
{ label: 'Samedi', short: 'SAM', value: 6 },
|
||||
{ label: 'Dimanche', short: 'DIM', value: 0 },
|
||||
]
|
||||
|
||||
type SlotKey = `${number}-${number}-${number}`
|
||||
|
||||
function toKey(s: ScheduleSlot): SlotKey {
|
||||
return `${s.day_of_week}-${s.hour}-${s.minute}`
|
||||
}
|
||||
|
||||
function fmt(h: number, m: number) {
|
||||
return `${String(h).padStart(2, '0')}:${String(m).padStart(2, '0')}`
|
||||
}
|
||||
|
||||
export function Schedule() {
|
||||
const [slots, setSlots] = useState<ScheduleSlot[]>([])
|
||||
const [loading, setLoading] = useState(true)
|
||||
const [saving, setSaving] = useState(false)
|
||||
const [saved, setSaved] = useState(false)
|
||||
const [newTimes, setNewTimes] = useState<Record<number, string>>({})
|
||||
|
||||
useEffect(() => { load() }, [])
|
||||
|
||||
async function load() {
|
||||
setLoading(true)
|
||||
try { setSlots((await adminApi.getSchedule()) ?? []) } finally { setLoading(false) }
|
||||
}
|
||||
|
||||
function slotsForDay(day: number) {
|
||||
return slots
|
||||
.filter(s => s.day_of_week === day)
|
||||
.sort((a, b) => a.hour !== b.hour ? a.hour - b.hour : a.minute - b.minute)
|
||||
}
|
||||
|
||||
function addSlot(day: number) {
|
||||
const time = newTimes[day] || '06:00'
|
||||
const [h, m] = time.split(':').map(Number)
|
||||
const newSlot: ScheduleSlot = { day_of_week: day, hour: h, minute: m }
|
||||
if (slots.some(s => toKey(s) === toKey(newSlot))) return
|
||||
setSlots(prev => [...prev, newSlot])
|
||||
setNewTimes(p => ({ ...p, [day]: '06:00' }))
|
||||
}
|
||||
|
||||
function removeSlot(slot: ScheduleSlot) {
|
||||
setSlots(prev => prev.filter(s => toKey(s) !== toKey(slot)))
|
||||
}
|
||||
|
||||
async function save() {
|
||||
setSaving(true); setSaved(false)
|
||||
await adminApi.updateSchedule(slots)
|
||||
setSaving(false); setSaved(true)
|
||||
setTimeout(() => setSaved(false), 2000)
|
||||
}
|
||||
|
||||
if (loading) return <div className="flex justify-center py-20"><Spinner /></div>
|
||||
|
||||
return (
|
||||
<div className="space-y-6">
|
||||
<div className="flex items-center justify-between">
|
||||
<div>
|
||||
<h1 className="text-2xl font-bold">Planning hebdomadaire</h1>
|
||||
<p className="text-muted-foreground text-sm">
|
||||
Définissez les créneaux de scraping + résumé IA pour chaque jour
|
||||
</p>
|
||||
</div>
|
||||
<Button onClick={save} disabled={saving}>
|
||||
{saving ? <Spinner className="h-4 w-4" /> : <Save className="h-4 w-4" />}
|
||||
{saved ? 'Enregistré !' : 'Enregistrer'}
|
||||
</Button>
|
||||
</div>
|
||||
|
||||
<div className="grid grid-cols-1 sm:grid-cols-2 lg:grid-cols-4 xl:grid-cols-7 gap-3">
|
||||
{DAYS.map(day => {
|
||||
const daySlots = slotsForDay(day.value)
|
||||
const isWeekend = day.value === 0 || day.value === 6
|
||||
return (
|
||||
<Card key={day.value} className={isWeekend ? 'border-muted' : ''}>
|
||||
<CardHeader className="pb-2 pt-4 px-4">
|
||||
<CardTitle className="text-sm font-semibold">
|
||||
<span className="hidden xl:block">{day.label}</span>
|
||||
<span className="xl:hidden">{day.short}</span>
|
||||
</CardTitle>
|
||||
</CardHeader>
|
||||
<CardContent className="px-4 pb-4 space-y-2">
|
||||
{/* Créneaux existants */}
|
||||
{daySlots.length === 0 && (
|
||||
<p className="text-xs text-muted-foreground italic">Aucun créneau</p>
|
||||
)}
|
||||
{daySlots.map(slot => (
|
||||
<div
|
||||
key={toKey(slot)}
|
||||
className="flex items-center justify-between rounded bg-primary/10 px-2 py-1"
|
||||
>
|
||||
<span className="text-sm font-mono font-medium">
|
||||
{fmt(slot.hour, slot.minute)}
|
||||
</span>
|
||||
<button
|
||||
onClick={() => removeSlot(slot)}
|
||||
className="text-muted-foreground hover:text-destructive transition-colors ml-2"
|
||||
>
|
||||
<Trash2 className="h-3 w-3" />
|
||||
</button>
|
||||
</div>
|
||||
))}
|
||||
|
||||
{/* Ajout d'un créneau */}
|
||||
<div className="flex items-center gap-1 pt-1">
|
||||
<input
|
||||
type="time"
|
||||
value={newTimes[day.value] ?? '06:00'}
|
||||
onChange={e => setNewTimes(p => ({ ...p, [day.value]: e.target.value }))}
|
||||
className="flex-1 min-w-0 rounded border border-input bg-background px-2 py-1 text-xs font-mono focus:outline-none focus:ring-1 focus:ring-ring"
|
||||
/>
|
||||
<button
|
||||
onClick={() => addSlot(day.value)}
|
||||
className="rounded bg-primary/10 p-1 hover:bg-primary/20 transition-colors"
|
||||
>
|
||||
<Plus className="h-3 w-3" />
|
||||
</button>
|
||||
</div>
|
||||
</CardContent>
|
||||
</Card>
|
||||
)
|
||||
})}
|
||||
</div>
|
||||
|
||||
<p className="text-xs text-muted-foreground">
|
||||
À chaque créneau, le service lance le scraping de toutes les sources actives puis génère les résumés IA.
|
||||
</p>
|
||||
</div>
|
||||
)
|
||||
}
|
||||
@ -34,12 +34,12 @@ export function Sources() {
|
||||
<div className="space-y-3">
|
||||
{sources.map(s => (
|
||||
<Card key={s.id}>
|
||||
<CardContent className="flex items-center justify-between py-4">
|
||||
<CardContent className="flex flex-col gap-3 py-4 sm:flex-row sm:items-center sm:justify-between">
|
||||
<div>
|
||||
<span className="font-semibold">{s.name}</span>
|
||||
<span className="ml-2 text-xs text-muted-foreground capitalize">({s.type})</span>
|
||||
</div>
|
||||
<div className="flex items-center gap-3">
|
||||
<div className="flex items-center gap-2">
|
||||
<Badge variant={s.enabled ? 'default' : 'outline'}>
|
||||
{s.enabled ? 'Activée' : 'Désactivée'}
|
||||
</Badge>
|
||||
|
||||
@ -15,8 +15,7 @@
|
||||
"noUnusedLocals": true,
|
||||
"noUnusedParameters": true,
|
||||
"noFallthroughCasesInSwitch": true,
|
||||
"baseUrl": ".",
|
||||
"paths": { "@/*": ["src/*"] }
|
||||
"paths": { "@/*": ["./src/*"] }
|
||||
},
|
||||
"include": ["src"]
|
||||
}
|
||||
|
||||
@ -16,9 +16,10 @@ export default defineConfig({
|
||||
theme_color: '#0f172a',
|
||||
background_color: '#0f172a',
|
||||
display: 'standalone',
|
||||
start_url: '/',
|
||||
icons: [
|
||||
{ src: '/icons/icon-192.png', sizes: '192x192', type: 'image/png' },
|
||||
{ src: '/icons/icon-512.png', sizes: '512x512', type: 'image/png' },
|
||||
{ src: '/icons/icon-192.png', sizes: '192x192', type: 'image/png', purpose: 'any' },
|
||||
{ src: '/icons/icon-512.png', sizes: '512x512', type: 'image/png', purpose: 'any maskable' },
|
||||
],
|
||||
},
|
||||
workbox: {
|
||||
|
||||
46
scraper-service/Dockerfile
Normal file
46
scraper-service/Dockerfile
Normal file
@ -0,0 +1,46 @@
|
||||
FROM node:20-slim
|
||||
|
||||
RUN apt-get update && apt-get install -y \
|
||||
chromium \
|
||||
fonts-liberation \
|
||||
libasound2 \
|
||||
libatk-bridge2.0-0 \
|
||||
libatk1.0-0 \
|
||||
libcairo2 \
|
||||
libcups2 \
|
||||
libdbus-1-3 \
|
||||
libexpat1 \
|
||||
libfontconfig1 \
|
||||
libgbm1 \
|
||||
libglib2.0-0 \
|
||||
libgtk-3-0 \
|
||||
libnspr4 \
|
||||
libnss3 \
|
||||
libpango-1.0-0 \
|
||||
libpangocairo-1.0-0 \
|
||||
libx11-6 \
|
||||
libx11-xcb1 \
|
||||
libxcb1 \
|
||||
libxcomposite1 \
|
||||
libxcursor1 \
|
||||
libxdamage1 \
|
||||
libxext6 \
|
||||
libxfixes3 \
|
||||
libxi6 \
|
||||
libxrandr2 \
|
||||
libxrender1 \
|
||||
libxss1 \
|
||||
libxtst6 \
|
||||
--no-install-recommends \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
ENV PUPPETEER_SKIP_CHROMIUM_DOWNLOAD=true
|
||||
ENV PUPPETEER_EXECUTABLE_PATH=/usr/bin/chromium
|
||||
|
||||
WORKDIR /app
|
||||
COPY package*.json ./
|
||||
RUN npm install --omit=dev
|
||||
COPY . .
|
||||
|
||||
EXPOSE 3001
|
||||
CMD ["node", "index.js"]
|
||||
205
scraper-service/index.js
Normal file
205
scraper-service/index.js
Normal file
@ -0,0 +1,205 @@
|
||||
const express = require('express')
|
||||
const puppeteer = require('puppeteer-extra')
|
||||
const StealthPlugin = require('puppeteer-extra-plugin-stealth')
|
||||
|
||||
puppeteer.use(StealthPlugin())
|
||||
|
||||
const app = express()
|
||||
app.use(express.json())
|
||||
|
||||
const CHROME_PATH = process.env.PUPPETEER_EXECUTABLE_PATH || '/usr/bin/chromium'
|
||||
const PORT = process.env.PORT || 3001
|
||||
|
||||
function launchBrowser() {
|
||||
return puppeteer.launch({
|
||||
executablePath: CHROME_PATH,
|
||||
headless: true,
|
||||
args: [
|
||||
'--no-sandbox',
|
||||
'--disable-setuid-sandbox',
|
||||
'--disable-dev-shm-usage',
|
||||
'--disable-gpu',
|
||||
'--window-size=1920,1080',
|
||||
'--disable-blink-features=AutomationControlled',
|
||||
],
|
||||
})
|
||||
}
|
||||
|
||||
async function tryClick(page, selectors) {
|
||||
for (const sel of selectors) {
|
||||
try {
|
||||
const el = await page.$(sel)
|
||||
if (el) { await el.click(); return true }
|
||||
} catch {}
|
||||
}
|
||||
await page.keyboard.press('Enter')
|
||||
return false
|
||||
}
|
||||
|
||||
async function tryType(page, selectors, text) {
|
||||
for (const sel of selectors) {
|
||||
try {
|
||||
await page.waitForSelector(sel, { timeout: 4000 })
|
||||
await page.type(sel, text, { delay: 60 })
|
||||
return true
|
||||
} catch {}
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
app.get('/health', (_, res) => res.json({ ok: true }))
|
||||
|
||||
app.post('/bloomberg/scrape', async (req, res) => {
|
||||
const { username, password } = req.body || {}
|
||||
if (!username || !password) {
|
||||
return res.status(400).json({ error: 'username and password required' })
|
||||
}
|
||||
|
||||
let browser
|
||||
try {
|
||||
browser = await launchBrowser()
|
||||
const page = await browser.newPage()
|
||||
await page.setViewport({ width: 1920, height: 1080 })
|
||||
|
||||
// Hide automation signals
|
||||
await page.evaluateOnNewDocument(() => {
|
||||
Object.defineProperty(navigator, 'webdriver', { get: () => undefined })
|
||||
window.chrome = { runtime: {} }
|
||||
Object.defineProperty(navigator, 'plugins', { get: () => [1, 2, 3] })
|
||||
})
|
||||
|
||||
console.log('[bloomberg] navigating to login page')
|
||||
await page.goto('https://www.bloomberg.com/account/signin', {
|
||||
waitUntil: 'networkidle2',
|
||||
timeout: 60000,
|
||||
})
|
||||
await new Promise(r => setTimeout(r, 2000))
|
||||
|
||||
// Debug: état de la page avant de chercher l'email
|
||||
const pageInputs = await page.evaluate(() =>
|
||||
Array.from(document.querySelectorAll('input')).map(i => ({
|
||||
type: i.type, name: i.name, id: i.id, placeholder: i.placeholder, visible: i.offsetParent !== null
|
||||
}))
|
||||
)
|
||||
console.log('[bloomberg] inputs on login page:', JSON.stringify(pageInputs))
|
||||
const pageTitle = await page.title()
|
||||
console.log('[bloomberg] page title:', pageTitle)
|
||||
|
||||
console.log('[bloomberg] entering email')
|
||||
const emailSelectors = [
|
||||
'#email-form-input',
|
||||
'input[id="email-form-input"]',
|
||||
'input[type="email"]',
|
||||
'input[name="text-input"]',
|
||||
'input[placeholder*="email" i]',
|
||||
]
|
||||
const emailOk = await tryType(page, emailSelectors, username)
|
||||
if (!emailOk) throw new Error('could not find email input')
|
||||
|
||||
await new Promise(r => setTimeout(r, 800))
|
||||
|
||||
// Click submit via JS pour contourner les boutons désactivés
|
||||
const submitted = await page.evaluate(() => {
|
||||
const btns = Array.from(document.querySelectorAll('button'))
|
||||
const btn = btns.find(b =>
|
||||
b.type === 'submit' ||
|
||||
/continue|next|sign.?in/i.test(b.textContent)
|
||||
)
|
||||
if (btn) { btn.click(); return true }
|
||||
const form = document.querySelector('form')
|
||||
if (form) { form.submit(); return true }
|
||||
return false
|
||||
})
|
||||
if (!submitted) await page.keyboard.press('Enter')
|
||||
|
||||
// Attendre que la page change (password input apparaît ou navigation)
|
||||
try {
|
||||
await page.waitForFunction(
|
||||
() => document.querySelector('input[type="password"]') !== null,
|
||||
{ timeout: 10000 }
|
||||
)
|
||||
} catch {
|
||||
await new Promise(r => setTimeout(r, 3000))
|
||||
}
|
||||
console.log('[bloomberg] after email submit, url:', page.url())
|
||||
|
||||
// Debug inputs disponibles
|
||||
const allInputs = await page.evaluate(() =>
|
||||
Array.from(document.querySelectorAll('input')).map(i => ({
|
||||
type: i.type, name: i.name, id: i.id, placeholder: i.placeholder
|
||||
}))
|
||||
)
|
||||
console.log('[bloomberg] inputs after email submit:', JSON.stringify(allInputs))
|
||||
|
||||
console.log('[bloomberg] entering password')
|
||||
const pwdOk = await tryType(page, [
|
||||
'input[type="password"]',
|
||||
'input[name="password"]',
|
||||
'input[autocomplete="current-password"]',
|
||||
'input[autocomplete="password"]',
|
||||
], password)
|
||||
if (!pwdOk) throw new Error('could not find password input — check logs above for available inputs')
|
||||
|
||||
await new Promise(r => setTimeout(r, 500))
|
||||
await tryClick(page, ['button[type="submit"]', 'input[type="submit"]'])
|
||||
await new Promise(r => setTimeout(r, 3000))
|
||||
|
||||
const currentURL = page.url()
|
||||
console.log('[bloomberg] after login, url:', currentURL)
|
||||
|
||||
const pages = [
|
||||
'https://www.bloomberg.com/markets',
|
||||
'https://www.bloomberg.com/technology',
|
||||
'https://www.bloomberg.com/economics',
|
||||
]
|
||||
|
||||
const articles = []
|
||||
const seen = new Set()
|
||||
|
||||
for (const url of pages) {
|
||||
try {
|
||||
console.log('[bloomberg] scraping', url)
|
||||
await page.goto(url, { waitUntil: 'networkidle2', timeout: 30000 })
|
||||
await new Promise(r => setTimeout(r, 2000))
|
||||
|
||||
const items = await page.evaluate(() => {
|
||||
const results = []
|
||||
const seen = new Set()
|
||||
const links = document.querySelectorAll(
|
||||
'a[href*="/news/articles"], a[href*="/opinion/"], a[href*="/markets/"]'
|
||||
)
|
||||
links.forEach(a => {
|
||||
if (seen.has(a.href)) return
|
||||
seen.add(a.href)
|
||||
const titleEl = a.querySelector('h1,h2,h3,h4,[class*="headline"],[class*="title"]')
|
||||
const text = titleEl ? titleEl.innerText.trim() : a.innerText.trim()
|
||||
if (text.length > 20 && a.href.includes('bloomberg.com')) {
|
||||
results.push({ title: text, url: a.href })
|
||||
}
|
||||
})
|
||||
return results.slice(0, 25)
|
||||
})
|
||||
|
||||
for (const item of items) {
|
||||
if (!seen.has(item.url) && item.title && item.url) {
|
||||
seen.add(item.url)
|
||||
articles.push(item)
|
||||
}
|
||||
}
|
||||
console.log('[bloomberg]', url, '->', items.length, 'articles')
|
||||
} catch (e) {
|
||||
console.error('[bloomberg] error on', url, ':', e.message)
|
||||
}
|
||||
}
|
||||
|
||||
console.log('[bloomberg] total:', articles.length, 'articles')
|
||||
res.json({ articles })
|
||||
} catch (e) {
|
||||
console.error('[bloomberg] scrape error:', e.message)
|
||||
res.status(500).json({ error: e.message })
|
||||
} finally {
|
||||
if (browser) await browser.close()
|
||||
}
|
||||
})
|
||||
|
||||
app.listen(PORT, () => console.log(`scraper-service listening on :${PORT}`))
|
||||
14
scraper-service/package.json
Normal file
14
scraper-service/package.json
Normal file
@ -0,0 +1,14 @@
|
||||
{
|
||||
"name": "tradarr-scraper-service",
|
||||
"version": "1.0.0",
|
||||
"main": "index.js",
|
||||
"scripts": {
|
||||
"start": "node index.js"
|
||||
},
|
||||
"dependencies": {
|
||||
"express": "^4.19.2",
|
||||
"puppeteer-extra": "^3.3.6",
|
||||
"puppeteer-extra-plugin-stealth": "^2.11.2",
|
||||
"puppeteer": "^22.0.0"
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user