Remove Lightning_Report_Automatic.json file, eliminating the n8n workflow and its components for report generation and token management.
This commit is contained in:
erdemerikci
parent
7db2a3daf3
commit
64b99eed06
File diff suppressed because it is too large
Load Diff
217
README.md
Normal file
217
README.md
Normal file
@ -0,0 +1,217 @@
|
||||
# Lightning Report (n8n)
|
||||
|
||||
Automated lightning activity reports for wind farms. An **n8n** workflow detects strikes via the [iklim.co](https://iklim.co) API, generates a Word report through a **Python report service**, routes it through **Slack** for human approval, and emails the approved report to customers via **SendGrid**.
|
||||
|
||||
## Architecture
|
||||
|
||||
```text
|
||||
┌─────────────┐ ┌──────────────────┐ ┌─────────────────────┐
|
||||
│ n8n workflow│────▶│ iklim.co API │ │ Report service │
|
||||
│ (scheduled) │ │ lightnings + │ │ FastAPI /generate │
|
||||
│ │ │ thunderstorms │ │ → DOCX │
|
||||
└──────┬──────┘ └──────────────────┘ └──────────▲──────────┘
|
||||
│ │
|
||||
│ Slack upload + approve/reject buttons │
|
||||
▼ │
|
||||
┌─────────────┐ storm_logs datatable (pending report) │
|
||||
│ Slack │───────────────────────────────────────────┘
|
||||
└──────┬──────┘
|
||||
│ on approve
|
||||
▼
|
||||
┌─────────────┐
|
||||
│ SendGrid │──▶ customer contact_email
|
||||
└─────────────┘
|
||||
```
|
||||
|
||||
## Repository layout
|
||||
|
||||
| Path | Description |
|
||||
|------|-------------|
|
||||
| `Lightning_Report_Automatic.json` | n8n workflow export (import in n8n) |
|
||||
| `report_service/` | FastAPI wrapper around report generation |
|
||||
| `report_service/adapter.py` | Maps n8n JSON payload → pandas DataFrames |
|
||||
| `src/reporting/docx.py` | Main DOCX builder |
|
||||
| `src/analysis/` | Risk, histogram, geospatial, statistics |
|
||||
| `src/visualization/` | Maps and storm-cell plots |
|
||||
| `src/config.py` | Default analysis parameters |
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- **Python 3.12+** (local or Docker)
|
||||
- **n8n** (self-hosted, with Data Tables enabled)
|
||||
- **iklim.co API** credentials (test: `https://api-test.iklim.co`)
|
||||
- **Slack** app/bot for uploads and approval buttons
|
||||
- **SendGrid** API key with Mail Send permission
|
||||
- Optional: **Gemini API key** if commentary is generated inside the report service (otherwise pass `gemini_text` from n8n)
|
||||
|
||||
## Report service (local)
|
||||
|
||||
### 1. Install dependencies
|
||||
|
||||
```bash
|
||||
cd Lightning_Report_n8n
|
||||
python -m venv .venv
|
||||
source .venv/bin/activate # Windows: .venv\Scripts\activate
|
||||
pip install -r requirements.txt -r report_service/requirements.txt
|
||||
```
|
||||
|
||||
### 2. Environment variables
|
||||
|
||||
Create a `.env` file in the project root (never commit it):
|
||||
|
||||
```env
|
||||
REPORT_SERVICE_TOKEN=your-long-random-secret
|
||||
GEMINI_API_KEY=optional-if-not-supplied-by-n8n
|
||||
GEMINI_MODEL=gemini-2.5-flash-lite
|
||||
LOG_LEVEL=INFO
|
||||
```
|
||||
|
||||
`REPORT_SERVICE_TOKEN` is required. The service refuses `/generate` if it is unset.
|
||||
|
||||
### 3. Run the server
|
||||
|
||||
```bash
|
||||
export REPORT_SERVICE_TOKEN="your-long-random-secret"
|
||||
uvicorn report_service.main:app --host 0.0.0.0 --port 8000
|
||||
```
|
||||
|
||||
Health check: `GET http://localhost:8000/health`
|
||||
|
||||
### 4. Expose to n8n (development)
|
||||
|
||||
If n8n runs elsewhere, tunnel the service:
|
||||
|
||||
```bash
|
||||
ngrok http 8000
|
||||
```
|
||||
|
||||
Point the n8n **Generate Report** HTTP Request node at `https://<your-ngrok-host>/generate` and set header `X-Report-Token` to the same value as `REPORT_SERVICE_TOKEN`.
|
||||
|
||||
## Report service (Docker)
|
||||
|
||||
```bash
|
||||
cd Lightning_Report_n8n
|
||||
export REPORT_SERVICE_TOKEN=your-long-random-secret
|
||||
docker compose -f report_service/docker-compose.yml up --build -d
|
||||
```
|
||||
|
||||
From n8n on the same Docker network: `http://report-service:8000/generate`
|
||||
|
||||
## API
|
||||
|
||||
### `GET /health`
|
||||
|
||||
Liveness probe. Returns `{"ok": true, ...}`.
|
||||
|
||||
### `POST /generate`
|
||||
|
||||
| Header | Required | Description |
|
||||
|--------|----------|-------------|
|
||||
| `X-Report-Token` | Yes | Must match `REPORT_SERVICE_TOKEN` |
|
||||
| `Content-Type` | Yes | `application/json` |
|
||||
|
||||
**Body** (built by n8n **Build Report Payload** node):
|
||||
|
||||
| Field | Description |
|
||||
|-------|-------------|
|
||||
| `customer_name` | Wind farm / customer label |
|
||||
| `t_start`, `t_end` | Storm window (epoch ms) |
|
||||
| `centroid_lat`, `centroid_lon` | Farm centroid |
|
||||
| `boundary_m` | Monitoring radius (m) |
|
||||
| `rings` | `{ r1, r2, r3, r4 }` distance rings |
|
||||
| `timezone` | e.g. `Europe/Istanbul` |
|
||||
| `turbines` | Array of turbine objects |
|
||||
| `strikes` | Lightning strike records (`captured` ISO time preferred) |
|
||||
| `storm_records` | Thunderstorm polygons from `/v1/thunderstorms/within` |
|
||||
| `gemini_text` | Optional pre-generated commentary |
|
||||
|
||||
**Response:** DOCX file (`application/vnd.openxmlformats-officedocument.wordprocessingml.document`).
|
||||
|
||||
## n8n workflow
|
||||
|
||||
Import `Lightning_Report_Automatic.json` into n8n and configure:
|
||||
|
||||
### Credentials
|
||||
|
||||
| Integration | Used for |
|
||||
|-------------|----------|
|
||||
| iklim.co login | `Login to iklim.co` / token refresh |
|
||||
| Slack (`n8n_lightning_report_bot`) | Report upload, approval buttons |
|
||||
| Slack (`Tarla Slack Account`) | Error / email-sent notifications |
|
||||
| SendGrid | Customer email after approval |
|
||||
|
||||
### Data tables
|
||||
|
||||
**`customers`**
|
||||
|
||||
| Column | Purpose |
|
||||
|--------|---------|
|
||||
| `customer_name` | Display name |
|
||||
| `contact_email` | SendGrid recipient |
|
||||
| `id` | Linked from wind turbines |
|
||||
|
||||
**`wind_turbine_farm`**
|
||||
|
||||
| Column | Purpose |
|
||||
|--------|---------|
|
||||
| `customer_id` | FK to customers |
|
||||
| `latitude`, `longitude` | Turbine positions |
|
||||
| `name` | Turbine label |
|
||||
|
||||
**`storm_logs`**
|
||||
|
||||
| Column | Purpose |
|
||||
|--------|---------|
|
||||
| `customer_name`, `storm_key`, `storm_start`, `storm_end` | Storm metadata |
|
||||
| `total_strikes`, `status` | Count; `waiting for confirmation` → `confirmed` / `rejected` |
|
||||
| `pending_contact_email` | Email snapshot at report time |
|
||||
| `pending_email_file_name` | DOCX filename |
|
||||
| `pending_report_base64` | Queued report (survives n8n restarts) |
|
||||
| `pending_report_mime_type` | MIME type for attachment |
|
||||
|
||||
### Main flow (simplified)
|
||||
|
||||
1. Daily trigger → authenticate → loop customers.
|
||||
2. Query lightnings within farm boundary; detect storm window.
|
||||
3. Insert/update `storm_logs`, fetch thunderstorms, build payload.
|
||||
4. **Generate Report** → upload DOCX to Slack → approval buttons.
|
||||
5. Persist pending report fields on `storm_logs`.
|
||||
6. On Slack **Approve** → load queued report → **SendGrid** email → clear pending fields.
|
||||
|
||||
Email is sent only after approval, not when the report is first generated.
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### `channel_not_found` (Slack)
|
||||
|
||||
The Slack channel ID in the node does not exist or the bot is not invited. Re-select the channel in n8n and run `/invite @YourBot` in that channel.
|
||||
|
||||
### `Account Is not Found For User …` (`/v1/thunderstorms/within`)
|
||||
|
||||
Login succeeded but the iklim user has no **account** on the API environment (e.g. test). Ask iklim.co to provision the user, or use a service account that has thunderstorm access. Lightning-only reports can continue if the workflow skips failed thunderstorm calls (optional branch).
|
||||
|
||||
### Histogram shows `01-01-1970` / wrong period
|
||||
|
||||
Strike timestamps were parsed incorrectly when `timestamp` (epoch ms) was preferred over `captured` (ISO). Fixed in `report_service/adapter.py` — restart the report service after pulling updates.
|
||||
|
||||
### SendGrid: attachment must be base64 encoded
|
||||
|
||||
Use the **Prepare SendGrid Email Payload** node path; ensure `pending_report_base64` in `storm_logs` is populated and not truncated by datatable size limits.
|
||||
|
||||
### Binary missing at **Upload Report to Slack**
|
||||
|
||||
The datatable persist step must run **after** Slack upload, not before (persist nodes drop binary data).
|
||||
|
||||
### Slack `invalid_payload` on approval follow-up
|
||||
|
||||
Do not double-`JSON.stringify` the body when posting to `response_url`. Slack response URLs expire after ~30 minutes (optional confirmation message only).
|
||||
|
||||
## Development notes
|
||||
|
||||
- Per-farm settings (`rings`, centroid, dates) come from the n8n payload via `apply_farm_config()`, not hardcoded in `src/config.py`.
|
||||
- Prefer strike field **`captured`** (ISO) for `local_time`; numeric **`timestamp`** is supported as epoch milliseconds.
|
||||
- HMAC signing for iklim API calls is implemented in n8n Code nodes (`Calculate Lightning Headers`, `Calculate Thunderstorm Headers`).
|
||||
|
||||
## License
|
||||
|
||||
Internal use — iklim.co lightning reporting pipeline.
|
||||
Loading…
x
Reference in New Issue
Block a user