Refactor: remove legacy CLI pipeline, simplify to n8n-driven service
This commit is contained in:
erdemerikci
parent
45d80dfaa6
commit
5d8c08dc86
@ -1,145 +0,0 @@
|
|||||||
# Batch Report Generation Guide
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
The batch generation system automates lightning report generation for multiple wind farms by fetching data from the API and processing them in batch.
|
|
||||||
|
|
||||||
## Setup
|
|
||||||
|
|
||||||
### 1. Install Dependencies
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install -r requirements.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
### 2. Create .env File
|
|
||||||
|
|
||||||
Create a `.env` file in the project root with your API key:
|
|
||||||
|
|
||||||
```env
|
|
||||||
API_KEY=your_api_key_here
|
|
||||||
```
|
|
||||||
|
|
||||||
### 3. Create Configuration File
|
|
||||||
|
|
||||||
Copy `wind_farms_config.example.json` to `wind_farms_config.json` and configure your wind farms:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cp wind_farms_config.example.json wind_farms_config.json
|
|
||||||
```
|
|
||||||
|
|
||||||
Edit `wind_farms_config.json` with your farms' details.
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
### API Configuration
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"api_config": {
|
|
||||||
"base_url": "https://risk.tarla.io/api",
|
|
||||||
"timeout_seconds": 30,
|
|
||||||
"retry_attempts": 3,
|
|
||||||
"default_query_range": {
|
|
||||||
"method": "current_month"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Wind Farm Configuration
|
|
||||||
|
|
||||||
Each farm can have:
|
|
||||||
|
|
||||||
- **enabled**: `true` or `false` - Controls whether to generate report
|
|
||||||
- **location_bounds.method**: `"auto"` (calculate from turbines) or `"manual"` (specify)
|
|
||||||
- **location_bounds.padding_km**: Extra buffer beyond max distance ring (default: 5km)
|
|
||||||
- **date_range.method**: `"auto"` (use query_range to fetch, then detect from data) or `"manual"` (specify dates)
|
|
||||||
|
|
||||||
### Query Range Options (for auto date_range)
|
|
||||||
|
|
||||||
- `"current_month"`: First day of current month to today
|
|
||||||
- `"last_month"`: Entire previous month
|
|
||||||
- `"days_back"`: Last N days (requires `days` parameter)
|
|
||||||
- `"custom"`: Specific dates (requires `start_date` and `end_date`)
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
### Process All Enabled Farms
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python batch_generate.py --config wind_farms_config.json
|
|
||||||
```
|
|
||||||
|
|
||||||
### List Farms
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python batch_generate.py --config wind_farms_config.json --list-farms
|
|
||||||
```
|
|
||||||
|
|
||||||
### Process Specific Farm
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python batch_generate.py --config wind_farms_config.json --farm-id dagpazari_RES
|
|
||||||
```
|
|
||||||
|
|
||||||
### Process All Farms (Ignore Enabled Flag)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python batch_generate.py --config wind_farms_config.json --force-all
|
|
||||||
```
|
|
||||||
|
|
||||||
### Process Disabled Farm
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python batch_generate.py --config wind_farms_config.json --farm-id disabled_farm --force
|
|
||||||
```
|
|
||||||
|
|
||||||
## How It Works
|
|
||||||
|
|
||||||
### Location Bounds Auto-Calculation
|
|
||||||
|
|
||||||
1. **Calculate Centroid**: Average of all turbine coordinates
|
|
||||||
2. **Find Max Distance**: Maximum distance from centroid to any turbine
|
|
||||||
3. **Add Distance Ring**: Add max distance ring (e.g., 30km)
|
|
||||||
4. **Add Padding**: Add padding (e.g., 5km)
|
|
||||||
5. **Result**: Center (centroid) + Radius (total distance)
|
|
||||||
|
|
||||||
## Output
|
|
||||||
|
|
||||||
- Reports are saved to each farm's `output_directory` in the config
|
|
||||||
- Batch summary saved to `reports/batch_summary_YYYY-MM-DD.json`
|
|
||||||
- Log file: `batch_generation_YYYY-MM-DD.log`
|
|
||||||
|
|
||||||
## API Endpoints
|
|
||||||
|
|
||||||
The system uses:
|
|
||||||
- Lightning data: `https://risk.tarla.io/api/lightning-data/historical/`
|
|
||||||
- Storm data: `https://risk.tarla.io/api/storm-data/historical/`
|
|
||||||
|
|
||||||
Parameters:
|
|
||||||
- `queryType=circle`
|
|
||||||
- `centerLongitude` (longitude first)
|
|
||||||
- `centerLatitude`
|
|
||||||
- `radius` (in meters)
|
|
||||||
- `startDate=YYYY-MM-DD`
|
|
||||||
- `endDate=YYYY-MM-DD`
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### API Key Not Found
|
|
||||||
|
|
||||||
Ensure `.env` file exists with `API_KEY=your_key`
|
|
||||||
|
|
||||||
### No Data Fetched
|
|
||||||
|
|
||||||
- Check API credentials
|
|
||||||
- Verify date range is correct
|
|
||||||
- Check if location bounds cover the area
|
|
||||||
- Verify API endpoint URLs
|
|
||||||
|
|
||||||
### Farm Skipped
|
|
||||||
|
|
||||||
- Check if `enabled: false` in config
|
|
||||||
- Use `--force` flag to process disabled farms
|
|
||||||
|
|
||||||
646
README.md
646
README.md
@ -1,646 +0,0 @@
|
|||||||
# Lightning Report Generator
|
|
||||||
|
|
||||||
A comprehensive Python application for analyzing lightning strike data in relation to wind turbine locations and generating detailed DOCX reports with risk assessments, visualizations, and statistical analysis.
|
|
||||||
|
|
||||||
## Overview
|
|
||||||
|
|
||||||
This application processes lightning strike data and wind turbine coordinates to:
|
|
||||||
- Calculate lightning risk scores for each turbine using advanced mathematical models
|
|
||||||
- Generate interactive maps showing lightning strikes and turbine locations
|
|
||||||
- Create statistical analysis and histograms with temporal distribution
|
|
||||||
- Group turbines based on proximity and risk levels
|
|
||||||
- Generate comprehensive DOCX reports with visualizations and risk assessment charts
|
|
||||||
- Support storm cell analysis and mapping
|
|
||||||
- Provide detailed risk score interpretation and calculation methodology
|
|
||||||
|
|
||||||
## Features
|
|
||||||
|
|
||||||
### Core Analysis
|
|
||||||
- **Risk Assessment**: Fast per-turbine scoring using BallTree radius queries (Haversine metric) with automatic fallback to vectorized matrix math
|
|
||||||
- **Advanced Risk Formula**: `Risk = P₀ × (1 + α×Current/10000) × e^(-α×Distance)` with configurable parameters
|
|
||||||
- **Geospatial Analysis**: Vectorized Haversine utilities and configurable distance rings
|
|
||||||
- **Statistical Analysis**: Lightning density, frequency, and temporal distribution analysis
|
|
||||||
- **Daily Lightning Density**: Calculates daily average using actual number of days in date range (not fixed month)
|
|
||||||
- **Turbine Grouping**: Proximity-based clustering using DBSCAN (Haversine) with graceful fallback to O(N^2) grouping for small datasets
|
|
||||||
|
|
||||||
### API Integration
|
|
||||||
- **Automated Data Fetching**: Fetch lightning and storm data directly from API
|
|
||||||
- **Flexible Location Bounds**: Auto-calculate center + radius from turbines or specify manually
|
|
||||||
- **Date Range Management**: Auto-detect actual period from data or use manual date ranges
|
|
||||||
- **Batch Processing**: Process multiple wind farms in a single run
|
|
||||||
- **Error Handling**: Graceful handling of empty data, API timeouts, and failures
|
|
||||||
|
|
||||||
### Visualization
|
|
||||||
- **Interactive Maps**: Plotly-based coordinate-plane maps for CG/IC lightning with ring-aware coloring
|
|
||||||
- **Risk Score Heatmap**: 2D visualization with current magnitude on X-axis (up to 300k amps) and distance on Y-axis, with contour curves
|
|
||||||
- **Fixed Interval Coloring**: Consistent color gradient mapping (blue to red) based on predefined risk score ranges (0.1-1.5)
|
|
||||||
- **Lightning Histograms**: Temporal distribution of lightning events with peak detection
|
|
||||||
- **Storm Cell Maps**: Visualization of storm cell data (when available)
|
|
||||||
- **Coordinate Plane Views**: Standard geographic orientation (latitude on Y-axis, longitude on X-axis)
|
|
||||||
|
|
||||||
### Reporting
|
|
||||||
- **DOCX Generation**: Word reports (DOCX)
|
|
||||||
- **Risk Score Chart**: Integrated heatmap showing distance vs. current magnitude relationship
|
|
||||||
- **Multiple Map Types**: Coordinate plane maps for different lightning types
|
|
||||||
- **Statistical Tables**: Detailed lightning strike information with proximity data (precomputed distances)
|
|
||||||
- **Risk Summaries**: Grouped risk analysis and recommendations with fixed interval color coding
|
|
||||||
- **Enhanced Appendix**: Detailed methodology explanations including risk calculation method, interpretation guide, and algorithm descriptions
|
|
||||||
|
|
||||||
### Data Processing
|
|
||||||
- **JSON Data Loading**: Support for various JSON data structures
|
|
||||||
- **Date Range Filtering**: Configurable analysis periods
|
|
||||||
- **Date/Time Formatting**: Centralized, consistent DD-MM-YYYY and DD-MM-YYYY HH:MM:SS formatting
|
|
||||||
- **Data Validation**: Comprehensive input validation and error handling
|
|
||||||
- **Precomputation**: Shared per-group distance and ring-index precompute reused by maps and tables
|
|
||||||
- **Coordinate Conversion**: UTM ED50 to WGS84 coordinate system conversion
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
### Prerequisites
|
|
||||||
- Python 3.8 or higher
|
|
||||||
- pip package manager
|
|
||||||
|
|
||||||
### Dependencies
|
|
||||||
Install the required packages:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
pip install -r requirements.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
### Required Packages
|
|
||||||
- `pandas>=1.5.0` - Data manipulation and analysis
|
|
||||||
- `numpy>=1.21.0` - Numerical computations
|
|
||||||
- `plotly>=5.15.0` - Interactive visualizations
|
|
||||||
- `kaleido>=0.2.1` - Static image export for Plotly
|
|
||||||
- `scikit-learn>=1.3.0` - BallTree radius queries and DBSCAN clustering (used when available)
|
|
||||||
- `requests>=2.31.0` - API HTTP requests
|
|
||||||
- `python-dotenv>=1.0.0` - Environment variable management
|
|
||||||
- `python-docx>=1.1.2` - DOCX (Word) report generation
|
|
||||||
|
|
||||||
### Optional Dependencies
|
|
||||||
For coordinate conversion functionality:
|
|
||||||
```bash
|
|
||||||
pip install -r utm_converter_requirements.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration
|
|
||||||
|
|
||||||
The application supports two modes of operation:
|
|
||||||
|
|
||||||
### 1. Single Report Generation (Legacy Mode)
|
|
||||||
|
|
||||||
Uses `src/config.py` for configuration. See the legacy section below for details.
|
|
||||||
|
|
||||||
### 2. Batch Report Generation (Recommended)
|
|
||||||
|
|
||||||
Uses `wind_farms_config.json` for multi-farm batch processing with API integration.
|
|
||||||
|
|
||||||
#### Setup
|
|
||||||
|
|
||||||
1. **Create `.env` file** with your API key:
|
|
||||||
```env
|
|
||||||
API_KEY=your_api_key_here
|
|
||||||
```
|
|
||||||
|
|
||||||
2. **Create `wind_farms_config.json`**:
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"api_config": {
|
|
||||||
"base_url": "https://risk.tarla.io/api",
|
|
||||||
"timeout_seconds": 30,
|
|
||||||
"retry_attempts": 3,
|
|
||||||
"default_query_range": {
|
|
||||||
"method": "current_month"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"output_base_directory": "reports/",
|
|
||||||
"default_padding_km": 5,
|
|
||||||
"wind_farms": [
|
|
||||||
{
|
|
||||||
"farm_id": "dagpazari_RES",
|
|
||||||
"name": "Dağpazarı RES",
|
|
||||||
"enabled": true,
|
|
||||||
"coordinates_file": "/path/to/coordinates.json",
|
|
||||||
"distance_rings": [1000, 2000, 3000, 4000, 10000],
|
|
||||||
"ring_colors": ["purple", "red", "orange", "coral", "green"],
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "auto",
|
|
||||||
"padding_km": 5
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "auto",
|
|
||||||
"query_range": {
|
|
||||||
"method": "current_month"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "reports/dagpazari_RES/",
|
|
||||||
"wind_farm_name": "Dağpazarı RES"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Configuration Parameters
|
|
||||||
|
|
||||||
**Farm-Level Settings:**
|
|
||||||
- `enabled`: `true`/`false` - Enable/disable report generation for this farm
|
|
||||||
- `distance_rings`: Array of distance rings in meters (e.g., `[1000, 2000, 3000, 4000, 10000]`)
|
|
||||||
- `ring_colors`: Array of colors for each ring
|
|
||||||
- `coordinates_file`: Path to turbine coordinates JSON file
|
|
||||||
|
|
||||||
**Location Bounds:**
|
|
||||||
- `method`: `"auto"` (calculate from turbines) or `"manual"` (specify)
|
|
||||||
- `padding_km`: Extra buffer beyond max distance ring (default: 5km)
|
|
||||||
- For manual: provide `center_lat`, `center_lng`, `radius_km`
|
|
||||||
|
|
||||||
**Date Range:**
|
|
||||||
- `method`: `"auto"` (detect from data) or `"manual"` (specify)
|
|
||||||
- For manual: provide `start_date` and `end_date` in `DD-MM-YYYY` format
|
|
||||||
- For auto: specify `query_range` to control API query period
|
|
||||||
|
|
||||||
**Query Range Options (for auto mode):**
|
|
||||||
- `"current_month"`: First day of current month to today
|
|
||||||
- `"last_month"`: Entire previous month
|
|
||||||
- `"days_back"`: Last N days (requires `days` parameter)
|
|
||||||
- `"custom"`: Specific dates (requires `start_date` and `end_date`)
|
|
||||||
|
|
||||||
#### Global Configuration (src/config.py)
|
|
||||||
|
|
||||||
The `src/config.py` file now only contains global defaults:
|
|
||||||
- Risk calculation parameters (`risk_params`)
|
|
||||||
- Histogram parameters (`histogram_params`)
|
|
||||||
- PDF layout parameters (`pdf_params`)
|
|
||||||
- Grouping parameters (`grouping_params`)
|
|
||||||
|
|
||||||
**Note:** Farm-specific settings (distance_rings, ring_colors, wind_farm_name, file paths, date ranges) are managed in `wind_farms_config.json` and should NOT be configured in `config.py`.
|
|
||||||
|
|
||||||
### Location Bounds Auto-Calculation
|
|
||||||
|
|
||||||
When `location_bounds.method = "auto"`, the system calculates:
|
|
||||||
|
|
||||||
1. **Centroid (Center Point)**:
|
|
||||||
- `center_lat` = average of all turbine latitudes
|
|
||||||
- `center_lng` = average of all turbine longitudes
|
|
||||||
|
|
||||||
2. **Maximum Distance from Centroid**:
|
|
||||||
- Calculates distance from centroid to each turbine
|
|
||||||
- Finds the maximum distance
|
|
||||||
|
|
||||||
3. **Total Radius**:
|
|
||||||
```
|
|
||||||
radius_km = (max_turbine_distance / 1000) +
|
|
||||||
(max_distance_ring / 1000) +
|
|
||||||
padding_km
|
|
||||||
```
|
|
||||||
|
|
||||||
Example: If turbines span 2.5km from centroid, max ring is 10km, padding is 5km:
|
|
||||||
- Total radius = 2.5 + 10 + 5 = 17.5km
|
|
||||||
|
|
||||||
### Date Range Handling
|
|
||||||
|
|
||||||
- If `date_range.method = "auto"`: Uses `query_range` to determine what dates to fetch; the report uses those query dates for the analyzed period.
|
|
||||||
- If `date_range.method = "manual"`: Uses specified `start_date` and `end_date` for both API fetch and report (supports `DD-MM-YYYY` or ISO with time, e.g. `2026-01-22T07:00:00Z`).
|
|
||||||
|
|
||||||
### Daily Lightning Density Calculation
|
|
||||||
|
|
||||||
The daily lightning density is calculated using the **actual number of days** in the analysis period:
|
|
||||||
|
|
||||||
```
|
|
||||||
daily_lightning_per_km2 = total_lightning_per_km2 / actual_days_in_range
|
|
||||||
```
|
|
||||||
|
|
||||||
Where `actual_days_in_range` is calculated from the start and end dates (inclusive).
|
|
||||||
|
|
||||||
**Example:**
|
|
||||||
- Date range: September 1-15 (15 days)
|
|
||||||
- Total lightning density: 150 events/km²
|
|
||||||
- Daily lightning density: 150 / 15 = 10 events/km²/day
|
|
||||||
|
|
||||||
This ensures accurate daily averages for partial months or custom date ranges.
|
|
||||||
|
|
||||||
### Risk Score Categories
|
|
||||||
The system uses fixed interval coloring based on specific risk score ranges:
|
|
||||||
- **Very Low Risk (<0.1)**: Blue - Distant lightning with low current
|
|
||||||
- **Low Risk (0.1-0.2)**: Teal - Moderate distance lightning
|
|
||||||
- **Med-Low Risk (0.2-0.4)**: Green - Closer lightning
|
|
||||||
- **Medium Risk (0.4-0.6)**: Yellow - Moderate risk lightning
|
|
||||||
- **Med-High Risk (0.6-0.8)**: Orange - High risk lightning
|
|
||||||
- **High Risk (0.8-1.0)**: Dark Orange - Very high risk lightning
|
|
||||||
- **Very High Risk (1.0-1.2)**: Red - Extreme risk lightning
|
|
||||||
- **Critical Risk (>1.2)**: Dark Red - Critical risk lightning
|
|
||||||
|
|
||||||
### Grouping vs Analysis Radius
|
|
||||||
- **grouping_params.max_distance_m (meters)**: Controls ONLY turbine clustering (grouping). If set (>0), it overrides ring-based grouping. Used to decide which turbines are in the same group.
|
|
||||||
- **grouping_params.distance_ring_index (0-based)**: Selects a ring from `distance_rings`.
|
|
||||||
- For grouping: used only if `max_distance_m` is not set; determines grouping radius.
|
|
||||||
- For analysis (histogram, stats, report labels): ALWAYS used to choose the analysis radius/cutoff. Does not change grouping when `max_distance_m` is provided.
|
|
||||||
|
|
||||||
Examples
|
|
||||||
- If `max_distance_m=2500` and `distance_ring_index=4` (10 km ring):
|
|
||||||
- Grouping radius = 2.5 km (from max_distance_m)
|
|
||||||
- Analysis radius = 10 km (from distance_ring_index)
|
|
||||||
- If `max_distance_m` unset and `distance_ring_index=1` (2 km ring):
|
|
||||||
- Grouping radius = 2 km
|
|
||||||
- Analysis radius = 2 km
|
|
||||||
|
|
||||||
Clustering Algorithm
|
|
||||||
- Preferred: DBSCAN with Haversine metric
|
|
||||||
- Convert lat/lng to radians; `eps = (radius_km / 6371)`, `min_samples=1`
|
|
||||||
- Clusters are formed transitively (density reachability). Example with R=2 km: A–B=1.5 km, B–C=1.5 km, A–C=3.0 km → one cluster {A,B,C} due to B bridging A and C
|
|
||||||
- Fallback: Greedy O(N^2) proximity grouping if scikit-learn is unavailable
|
|
||||||
- Starts a group at turbine i; adds any j within R of i; moves on. No transitive chaining
|
|
||||||
|
|
||||||
### Wind Farm Configuration
|
|
||||||
```python
|
|
||||||
wind_farm_name = "Your Wind Farm Name"
|
|
||||||
```
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
### Batch Report Generation (Recommended)
|
|
||||||
|
|
||||||
Generate reports for multiple wind farms automatically:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# Process all enabled farms
|
|
||||||
python batch_generate.py --config wind_farms_config.json
|
|
||||||
|
|
||||||
# Process specific farm
|
|
||||||
python batch_generate.py --config wind_farms_config.json --farm-id dagpazari_RES
|
|
||||||
|
|
||||||
# List farms and their enabled status
|
|
||||||
python batch_generate.py --config wind_farms_config.json --list-farms
|
|
||||||
|
|
||||||
# Process all farms (ignore enabled flag)
|
|
||||||
python batch_generate.py --config wind_farms_config.json --force-all
|
|
||||||
```
|
|
||||||
|
|
||||||
The batch system will:
|
|
||||||
1. Load configuration from `wind_farms_config.json`
|
|
||||||
2. For each enabled farm:
|
|
||||||
- Load turbine coordinates
|
|
||||||
- Auto-calculate location bounds (center + radius) from turbines
|
|
||||||
- Determine date range for API query
|
|
||||||
- Fetch lightning data from API
|
|
||||||
- Fetch storm data from API
|
|
||||||
- Calculate risk scores
|
|
||||||
- Generate DOCX report
|
|
||||||
- Save to farm's output directory
|
|
||||||
3. Generate batch summary report
|
|
||||||
|
|
||||||
### Single Report Generation (Legacy)
|
|
||||||
|
|
||||||
Run the main application for a single report:
|
|
||||||
```bash
|
|
||||||
python main.py
|
|
||||||
```
|
|
||||||
|
|
||||||
The application will:
|
|
||||||
1. Load lightning and turbine data from configured JSON files (in `src/config.py`)
|
|
||||||
2. Calculate risk scores for each turbine using the advanced risk formula
|
|
||||||
3. Create turbine groups based on proximity
|
|
||||||
4. Generate visualizations including the new risk score heatmap
|
|
||||||
5. Create a comprehensive DOCX report with enhanced appendix
|
|
||||||
|
|
||||||
### Data Format Requirements
|
|
||||||
|
|
||||||
#### Lightning Data JSON
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"data": [
|
|
||||||
{
|
|
||||||
"lat": 39.85420,
|
|
||||||
"lng": 26.71218,
|
|
||||||
"local_time": "2025-07-15T14:30:25",
|
|
||||||
"current": -15000,
|
|
||||||
"p_type": "0",
|
|
||||||
"height": 5000
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Required Fields:**
|
|
||||||
- `lat`, `lng`: Lightning strike coordinates
|
|
||||||
- `local_time`: Timestamp (various formats supported)
|
|
||||||
- `current`: Lightning current in amperes
|
|
||||||
- `p_type`: Lightning type ("0" for cloud-to-ground, others for intercloud)
|
|
||||||
|
|
||||||
#### Turbine Data JSON
|
|
||||||
```json
|
|
||||||
[
|
|
||||||
{
|
|
||||||
"lat": 39.85420,
|
|
||||||
"lng": 26.71218,
|
|
||||||
"turbine_id": "T001"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
```
|
|
||||||
|
|
||||||
**Required Fields:**
|
|
||||||
- `lat`, `lng`: Turbine coordinates
|
|
||||||
- `turbine_id`: Unique turbine identifier
|
|
||||||
|
|
||||||
### Advanced Usage
|
|
||||||
|
|
||||||
#### Coordinate Conversion
|
|
||||||
Convert UTM ED50 coordinates to WGS84:
|
|
||||||
```bash
|
|
||||||
python utm_ed50_to_wgs84_converter.py input.csv output.csv
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Data Separation by Month
|
|
||||||
Separate large JSON files by month:
|
|
||||||
```bash
|
|
||||||
python separate_by_month.py input_data.json [output_directory]
|
|
||||||
```
|
|
||||||
|
|
||||||
## Output
|
|
||||||
|
|
||||||
### DOCX Report Structure
|
|
||||||
1. **Cover Page**: Wind farm information and analysis period
|
|
||||||
2. **Report Summary**: Automated narrative summary (Gemini-backed when available)
|
|
||||||
3. **Risk Analysis**: Detailed risk scores and rankings with fixed interval coloring
|
|
||||||
4. **Lightning Maps**: Coordinate plane visualizations with proper geographic orientation
|
|
||||||
5. **Statistical Analysis**: Lightning density and frequency data
|
|
||||||
6. **Detailed Tables**: Complete lightning strike information with color-coded distance rings
|
|
||||||
7. **Storm Analysis**: Storm cell data and maps (if available)
|
|
||||||
8. **Enhanced Appendix**: Comprehensive methodology including:
|
|
||||||
- Risk calculation method and formula explanation
|
|
||||||
- Risk score interpretation guide
|
|
||||||
- Centroid and distance ring calculation methodology
|
|
||||||
- Turbine grouping algorithm description
|
|
||||||
- Frequent lightning activity period detection algorithm
|
|
||||||
|
|
||||||
### Generated Files
|
|
||||||
|
|
||||||
**Single Report Mode:**
|
|
||||||
- `lightning_report.log`: Application execution log
|
|
||||||
- `{wind_farm_name}_lightning_report.docx`: Main DOCX report
|
|
||||||
- Interactive HTML maps (temporary files)
|
|
||||||
|
|
||||||
**Batch Generation Mode:**
|
|
||||||
- `batch_generation_YYYY-MM-DD.log`: Batch execution log
|
|
||||||
- `batch_summary_YYYY-MM-DD.json`: Batch processing summary
|
|
||||||
- `{farm_id}_report.docx`: DOCX report for each farm (in respective output directories)
|
|
||||||
|
|
||||||
## Project Structure
|
|
||||||
|
|
||||||
```
|
|
||||||
lightning_report/
|
|
||||||
├── main.py # Single report generation (legacy)
|
|
||||||
├── batch_generate.py # Batch report generation with API
|
|
||||||
├── wind_farms_config.json # Batch configuration file
|
|
||||||
├── .env # API credentials (gitignored)
|
|
||||||
├── requirements.txt # Python dependencies
|
|
||||||
├── src/
|
|
||||||
│ ├── config.py # Global configuration defaults
|
|
||||||
│ ├── api/
|
|
||||||
│ │ └── data_fetcher.py # API integration for data fetching
|
|
||||||
│ ├── data/
|
|
||||||
│ │ └── loader.py # Data loading and validation
|
|
||||||
│ ├── analysis/
|
|
||||||
│ │ ├── geospatial.py # Distance calculations (vectorized Haversine)
|
|
||||||
│ │ ├── grouping.py # Turbine grouping (DBSCAN + fallback)
|
|
||||||
│ │ ├── histogram.py # Temporal analysis
|
|
||||||
│ │ ├── risk.py # Risk calculation (BallTree + fallback)
|
|
||||||
│ │ └── statistics.py # Statistical analysis (includes daily density)
|
|
||||||
│ ├── reporting/
|
|
||||||
│ │ ├── docx.py # DOCX report generation
|
|
||||||
│ │ ├── docx_sections.py # Shared DOCX helpers (charts/tables)
|
|
||||||
│ │ └── precompute.py # Shared precomputations (distances, ring indices)
|
|
||||||
│ ├── visualization/
|
|
||||||
│ │ ├── maps.py # Map generation with risk score heatmap
|
|
||||||
│ │ └── storm_cells.py # Storm cell visualization
|
|
||||||
│ └── utils.py # Utility functions including fixed interval coloring
|
|
||||||
├── separate_by_month.py # Data separation utility
|
|
||||||
└── utm_ed50_to_wgs84_converter.py # Coordinate conversion
|
|
||||||
```
|
|
||||||
|
|
||||||
## Configuration Examples
|
|
||||||
|
|
||||||
### Batch Generation Setup
|
|
||||||
|
|
||||||
**Example: Multiple Farms with Different Settings**
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"api_config": {
|
|
||||||
"base_url": "https://risk.tarla.io/api",
|
|
||||||
"timeout_seconds": 30,
|
|
||||||
"retry_attempts": 3
|
|
||||||
},
|
|
||||||
"wind_farms": [
|
|
||||||
{
|
|
||||||
"farm_id": "farm1",
|
|
||||||
"name": "Farm 1",
|
|
||||||
"enabled": true,
|
|
||||||
"coordinates_file": "/path/to/farm1_coordinates.json",
|
|
||||||
"distance_rings": [1000, 2000, 3000, 4000, 10000],
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "auto",
|
|
||||||
"padding_km": 5
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "manual",
|
|
||||||
"start_date": "01-09-2025",
|
|
||||||
"end_date": "30-09-2025"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "reports/farm1/",
|
|
||||||
"wind_farm_name": "Farm 1"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"farm_id": "farm2",
|
|
||||||
"name": "Farm 2",
|
|
||||||
"enabled": false,
|
|
||||||
"coordinates_file": "/path/to/farm2_coordinates.json",
|
|
||||||
"distance_rings": [1000, 2000, 3000, 4000, 10000],
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "manual",
|
|
||||||
"center_lat": 36.90,
|
|
||||||
"center_lng": 33.575,
|
|
||||||
"radius_km": 35
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "auto",
|
|
||||||
"query_range": {
|
|
||||||
"method": "days_back",
|
|
||||||
"days": 30
|
|
||||||
}
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "reports/farm2/",
|
|
||||||
"wind_farm_name": "Farm 2"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
### Custom Risk Parameters
|
|
||||||
```python
|
|
||||||
# Adjust risk calculation sensitivity in src/config.py
|
|
||||||
risk_params = {
|
|
||||||
'P_0': 1.5, # Higher base probability
|
|
||||||
'alpha': 0.3, # Slower distance decay
|
|
||||||
'current_weight': 0.2 # Higher current importance
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
**Note:** Farm-specific settings (distance_rings, ring_colors, etc.) should be configured in `wind_farms_config.json`, not in `config.py`.
|
|
||||||
|
|
||||||
## Risk Score Methodology
|
|
||||||
|
|
||||||
### Risk Calculation Formula
|
|
||||||
The system uses an advanced risk calculation formula:
|
|
||||||
```
|
|
||||||
Risk = P₀ × (1 + α×Current/10000) × e^(-α×Distance)
|
|
||||||
```
|
|
||||||
|
|
||||||
Where:
|
|
||||||
- **P₀**: Base probability (configurable)
|
|
||||||
- **α**: Distance decay factor (configurable)
|
|
||||||
- **Current**: Lightning current magnitude in amperes
|
|
||||||
- **Distance**: Distance from turbine in kilometers
|
|
||||||
|
|
||||||
### Risk Score Interpretation
|
|
||||||
The risk score heatmap provides a visual reference for interpreting risk levels:
|
|
||||||
- **X-axis**: Lightning current magnitude (1,000 to 300,000 amperes)
|
|
||||||
- **Y-axis**: Distance from turbine (0.1 km to max distance ring, dynamically scaled)
|
|
||||||
- **Color intensity**: Risk score level (blue to red gradient using palette: F94144, F3722C, F8961E, F9C74F, 90BE6D, 43AA8B, 577590)
|
|
||||||
- **Contour curves**: Specific risk level boundaries (0.1, 0.2, 0.4, 0.6, 0.8, 1.0, 1.2, 1.4, 1.5)
|
|
||||||
|
|
||||||
### API Integration
|
|
||||||
|
|
||||||
The system integrates with the Tarla.io API for automated data fetching:
|
|
||||||
|
|
||||||
**Endpoints:**
|
|
||||||
- Lightning data: `https://risk.tarla.io/api/lightning-data/historical/`
|
|
||||||
- Storm data: `https://risk.tarla.io/api/storm-data/historical/`
|
|
||||||
|
|
||||||
**Authentication:**
|
|
||||||
- API key stored in `.env` file as `API_KEY`
|
|
||||||
- Sent as `x-api-key` header in requests
|
|
||||||
|
|
||||||
**Request Format:**
|
|
||||||
- Query type: `circle` (center + radius)
|
|
||||||
- Parameters: `centerLatitude`, `centerLongitude`, `radius` (in meters), `startDate`, `endDate`
|
|
||||||
- Date format: `YYYY-MM-DD`
|
|
||||||
|
|
||||||
**Response Handling:**
|
|
||||||
- Automatically converts API responses to expected DataFrame format
|
|
||||||
- Handles empty datasets gracefully
|
|
||||||
- Validates data structure before processing
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Common Issues
|
|
||||||
|
|
||||||
1. **API Authentication Errors (401 Unauthorized)**
|
|
||||||
- Verify `.env` file exists with `API_KEY=your_key`
|
|
||||||
- Check that API key is correct and active
|
|
||||||
- Ensure API key contains special characters correctly (e.g., `==` at the end)
|
|
||||||
|
|
||||||
2. **API Timeout Errors**
|
|
||||||
- Increase `timeout_seconds` in `api_config`
|
|
||||||
- Check network connectivity
|
|
||||||
- Verify API endpoint is accessible
|
|
||||||
|
|
||||||
3. **File Not Found Errors**
|
|
||||||
- For batch mode: Verify file paths in `wind_farms_config.json`
|
|
||||||
- For single mode: Verify file paths in `src/config.py`
|
|
||||||
- Ensure JSON files exist and are readable
|
|
||||||
|
|
||||||
4. **Data Validation Errors**
|
|
||||||
- Check JSON format matches required structure
|
|
||||||
- Verify coordinate values are valid numbers
|
|
||||||
- Ensure timestamp format is supported
|
|
||||||
- For API data: Check API response format matches expected structure
|
|
||||||
|
|
||||||
5. **Empty Data / NaT Errors**
|
|
||||||
- System handles empty datasets gracefully
|
|
||||||
- Check API date range - data might not exist for specified period
|
|
||||||
- Verify location bounds cover the area of interest
|
|
||||||
- Check logs for API response details
|
|
||||||
|
|
||||||
6. **Memory Issues with Large Datasets**
|
|
||||||
- Use `separate_by_month.py` to split large files
|
|
||||||
- Adjust analysis period to smaller time ranges
|
|
||||||
- Process farms individually using `--farm-id` flag
|
|
||||||
|
|
||||||
7. **DOCX Generation Errors**
|
|
||||||
- Ensure sufficient disk space
|
|
||||||
- Check write permissions for output directory
|
|
||||||
|
|
||||||
8. **Risk Score Heatmap Issues**
|
|
||||||
- Verify distance_rings configuration is valid
|
|
||||||
- Check that lightning data contains valid current values
|
|
||||||
- Ensure turbine coordinates are properly formatted
|
|
||||||
|
|
||||||
9. **Batch Generation Issues**
|
|
||||||
- Check `batch_summary_YYYY-MM-DD.json` for detailed error information
|
|
||||||
- Verify all farms have valid configuration
|
|
||||||
- Check `batch_generation_YYYY-MM-DD.log` for detailed logs
|
|
||||||
- Use `--list-farms` to verify farm configuration
|
|
||||||
|
|
||||||
### Logging
|
|
||||||
|
|
||||||
**Single Report Mode:**
|
|
||||||
- `lightning_report.log`: Application execution log
|
|
||||||
|
|
||||||
**Batch Generation Mode:**
|
|
||||||
- `batch_generation_YYYY-MM-DD.log`: Batch execution log with per-farm details
|
|
||||||
- `batch_summary_YYYY-MM-DD.json`: Structured summary of batch processing
|
|
||||||
|
|
||||||
Logs include:
|
|
||||||
- Data loading progress
|
|
||||||
- API request/response details
|
|
||||||
- Risk calculation details
|
|
||||||
- Error messages and stack traces
|
|
||||||
- Performance metrics
|
|
||||||
- Farm processing status
|
|
||||||
|
|
||||||
## Performance Considerations
|
|
||||||
|
|
||||||
- **Large Datasets**: For datasets with >100,000 lightning strikes, consider:
|
|
||||||
- Using date range filtering
|
|
||||||
- Splitting data by month
|
|
||||||
- Increasing system memory allocation
|
|
||||||
|
|
||||||
- **Optimizations used**:
|
|
||||||
- BallTree neighbor queries for CG risk scoring (O(n log n) build; sublinear queries)
|
|
||||||
- DBSCAN clustering with Haversine metric for grouping; O(N^2) fallback maintained
|
|
||||||
- Vectorized Haversine distance utilities (array-based)
|
|
||||||
- Shared per-group precomputation of distances and ring indices reused by maps and tables
|
|
||||||
- Centralized date/time parsing and formatting
|
|
||||||
- Efficient risk score heatmap generation with contour overlay
|
|
||||||
|
|
||||||
## Contributing
|
|
||||||
|
|
||||||
1. Follow the existing code structure and naming conventions
|
|
||||||
2. Add appropriate error handling and logging
|
|
||||||
3. Update configuration options as needed
|
|
||||||
4. Test with various data formats and sizes
|
|
||||||
5. Update documentation for new features
|
|
||||||
6. Maintain consistency with the fixed interval coloring system
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
This project is proprietary software. All rights reserved.
|
|
||||||
|
|
||||||
## Support
|
|
||||||
|
|
||||||
For technical support or feature requests, please contact the development team with:
|
|
||||||
- Detailed error messages
|
|
||||||
- Sample data (if possible)
|
|
||||||
- System configuration details
|
|
||||||
- Expected vs actual behavior description
|
|
||||||
@ -1,181 +0,0 @@
|
|||||||
# UTM ED50 to WGS84 Coordinate Converter
|
|
||||||
|
|
||||||
This script converts UTM (Universal Transverse Mercator) coordinates from ED50 (European Datum 1950) reference system to WGS84 format. It supports 6-degree UTM zones and handles both northern and southern hemispheres.
|
|
||||||
|
|
||||||
## Features
|
|
||||||
|
|
||||||
- Convert single coordinates interactively
|
|
||||||
- Batch convert coordinates from CSV files
|
|
||||||
- Support for all UTM zones (1-60)
|
|
||||||
- Automatic handling of northern/southern hemispheres
|
|
||||||
- Error handling and validation
|
|
||||||
- Detailed conversion statistics
|
|
||||||
|
|
||||||
## Installation
|
|
||||||
|
|
||||||
1. Install the required dependencies:
|
|
||||||
```bash
|
|
||||||
pip install -r utm_converter_requirements.txt
|
|
||||||
```
|
|
||||||
|
|
||||||
Or install manually:
|
|
||||||
```bash
|
|
||||||
pip install pyproj pandas
|
|
||||||
```
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
### Interactive Mode
|
|
||||||
|
|
||||||
Run the script in interactive mode to convert single coordinates:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python utm_ed50_to_wgs84_converter.py --interactive
|
|
||||||
```
|
|
||||||
|
|
||||||
Example session:
|
|
||||||
```
|
|
||||||
UTM ED50 to WGS84 Coordinate Converter
|
|
||||||
========================================
|
|
||||||
Enter coordinates (type 'quit' to exit)
|
|
||||||
|
|
||||||
Enter easting (meters): 500000
|
|
||||||
Enter northing (meters): 4500000
|
|
||||||
Enter UTM zone (1-60): 35
|
|
||||||
Enter hemisphere (N/S) [default: N]: N
|
|
||||||
|
|
||||||
WGS84 Coordinates:
|
|
||||||
Latitude: 40.12345678°
|
|
||||||
Longitude: 32.87654321°
|
|
||||||
----------------------------------------
|
|
||||||
```
|
|
||||||
|
|
||||||
### Batch Conversion from CSV
|
|
||||||
|
|
||||||
Convert multiple coordinates from a CSV file:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python utm_ed50_to_wgs84_converter.py input.csv output.csv
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Input CSV Format
|
|
||||||
|
|
||||||
The input CSV should contain the following columns:
|
|
||||||
|
|
||||||
| Column | Description | Required | Default |
|
|
||||||
|--------|-------------|----------|---------|
|
|
||||||
| `easting` | UTM easting coordinate in meters | Yes | - |
|
|
||||||
| `northing` | UTM northing coordinate in meters | Yes | - |
|
|
||||||
| `zone` | UTM zone number (1-60) | Yes | - |
|
|
||||||
| `northern` | Hemisphere flag (True/False) | No | True |
|
|
||||||
|
|
||||||
Example input CSV:
|
|
||||||
```csv
|
|
||||||
easting,northing,zone,northern,description
|
|
||||||
500000,4500000,35,True,Sample point 1
|
|
||||||
600000,4600000,36,True,Sample point 2
|
|
||||||
400000,4400000,34,True,Sample point 3
|
|
||||||
```
|
|
||||||
|
|
||||||
#### Output CSV Format
|
|
||||||
|
|
||||||
The output CSV will contain all original columns plus:
|
|
||||||
- `wgs84_lat`: WGS84 latitude in decimal degrees
|
|
||||||
- `wgs84_lon`: WGS84 longitude in decimal degrees
|
|
||||||
|
|
||||||
### Custom Column Names
|
|
||||||
|
|
||||||
If your CSV uses different column names, specify them with command line arguments:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python utm_ed50_to_wgs84_converter.py input.csv output.csv \
|
|
||||||
--easting-col X \
|
|
||||||
--northing-col Y \
|
|
||||||
--zone-col ZONE \
|
|
||||||
--northern-col HEMISPHERE
|
|
||||||
```
|
|
||||||
|
|
||||||
## Technical Details
|
|
||||||
|
|
||||||
### Coordinate Systems
|
|
||||||
|
|
||||||
- **ED50 (European Datum 1950)**: Historical European geodetic datum
|
|
||||||
- **WGS84**: World Geodetic System 1984, current global standard
|
|
||||||
- **UTM**: Universal Transverse Mercator projection system
|
|
||||||
|
|
||||||
### Conversion Process
|
|
||||||
|
|
||||||
1. **ED50 UTM → WGS84 UTM**: Transform between datums using pyproj
|
|
||||||
2. **WGS84 UTM → WGS84 Lat/Lon**: Convert from projected to geographic coordinates
|
|
||||||
|
|
||||||
### UTM Zones
|
|
||||||
|
|
||||||
The script supports all 60 UTM zones:
|
|
||||||
- Zones 1-60 cover the globe in 6-degree longitude bands
|
|
||||||
- Zone 1: 180°W to 174°W
|
|
||||||
- Zone 60: 174°E to 180°E
|
|
||||||
|
|
||||||
### Accuracy
|
|
||||||
|
|
||||||
The conversion accuracy depends on:
|
|
||||||
- Quality of the original ED50 coordinates
|
|
||||||
- Geographic location (accuracy varies by region)
|
|
||||||
- Typically within 1-10 meters for most European locations
|
|
||||||
|
|
||||||
## Examples
|
|
||||||
|
|
||||||
### Example 1: Interactive Conversion
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python utm_ed50_to_wgs84_converter.py --interactive
|
|
||||||
```
|
|
||||||
|
|
||||||
### Example 2: Batch Conversion
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python utm_ed50_to_wgs84_converter.py sample_utm_ed50_data.csv converted_coordinates.csv
|
|
||||||
```
|
|
||||||
|
|
||||||
### Example 3: Custom Column Names
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python utm_ed50_to_wgs84_converter.py data.csv output.csv \
|
|
||||||
--easting-col X_COORD \
|
|
||||||
--northing-col Y_COORD \
|
|
||||||
--zone-col UTM_ZONE
|
|
||||||
```
|
|
||||||
|
|
||||||
## Error Handling
|
|
||||||
|
|
||||||
The script handles various error conditions:
|
|
||||||
|
|
||||||
- **Invalid UTM zones**: Must be between 1-60
|
|
||||||
- **Missing columns**: Reports which required columns are missing
|
|
||||||
- **Invalid coordinates**: Skips invalid rows and reports warnings
|
|
||||||
- **File not found**: Clear error messages for missing input files
|
|
||||||
|
|
||||||
## Dependencies
|
|
||||||
|
|
||||||
- **pyproj**: Coordinate transformation library
|
|
||||||
- **pandas**: Data manipulation and CSV handling
|
|
||||||
- **argparse**: Command line argument parsing
|
|
||||||
|
|
||||||
## License
|
|
||||||
|
|
||||||
This script is provided as-is for educational and practical use.
|
|
||||||
|
|
||||||
## Troubleshooting
|
|
||||||
|
|
||||||
### Common Issues
|
|
||||||
|
|
||||||
1. **"Invalid UTM zone" error**: Ensure zone numbers are between 1-60
|
|
||||||
2. **"Missing required columns" error**: Check your CSV column names
|
|
||||||
3. **Conversion failures**: Verify coordinate values are numeric
|
|
||||||
4. **Import errors**: Install required dependencies with pip
|
|
||||||
|
|
||||||
### Getting Help
|
|
||||||
|
|
||||||
Run the script with `--help` for command line options:
|
|
||||||
```bash
|
|
||||||
python utm_ed50_to_wgs84_converter.py --help
|
|
||||||
```
|
|
||||||
@ -1,463 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""
|
|
||||||
Batch generate lightning reports for multiple wind farms.
|
|
||||||
"""
|
|
||||||
|
|
||||||
import sys
|
|
||||||
import json
|
|
||||||
import argparse
|
|
||||||
import logging
|
|
||||||
from datetime import datetime
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
import pandas as pd
|
|
||||||
from dotenv import load_dotenv
|
|
||||||
|
|
||||||
from src.api.data_fetcher import APIDataFetcher
|
|
||||||
from src.data.loader import load_turbine_data, load_lightning_data_from_csv
|
|
||||||
from src.analysis.risk import calculate_turbine_risks
|
|
||||||
from src.analysis.grouping import create_turbine_groups
|
|
||||||
from src.reporting.docx import create_docx_report
|
|
||||||
from src.reporting.filename_utils import farm_local_date_range_from_config, slugify_ascii_underscore
|
|
||||||
from src.utils import (
|
|
||||||
filter_lightning_data_by_date_range,
|
|
||||||
format_date_ddmmyyyy,
|
|
||||||
format_period_display_for_report,
|
|
||||||
normalize_local_time_to_timezone,
|
|
||||||
)
|
|
||||||
from src.config import config as global_config
|
|
||||||
|
|
||||||
load_dotenv()
|
|
||||||
|
|
||||||
logging.basicConfig(
|
|
||||||
level=logging.INFO,
|
|
||||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
|
|
||||||
handlers=[
|
|
||||||
logging.StreamHandler(sys.stdout),
|
|
||||||
logging.FileHandler(f'batch_generation_{datetime.now().strftime("%Y-%m-%d")}.log')
|
|
||||||
]
|
|
||||||
)
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
def load_wind_farms_config(config_path: str) -> dict:
|
|
||||||
"""Load wind farms configuration from JSON file."""
|
|
||||||
try:
|
|
||||||
with open(config_path, 'r', encoding='utf-8') as f:
|
|
||||||
config = json.load(f)
|
|
||||||
logger.info(f"Loaded configuration from {config_path}")
|
|
||||||
return config
|
|
||||||
except FileNotFoundError:
|
|
||||||
logger.error(f"Configuration file not found: {config_path}")
|
|
||||||
raise
|
|
||||||
except json.JSONDecodeError as e:
|
|
||||||
logger.error(f"Invalid JSON in configuration file: {e}")
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
def filter_enabled_farms(wind_farms: list) -> tuple:
|
|
||||||
"""Filter farms by enabled status."""
|
|
||||||
enabled = []
|
|
||||||
disabled = []
|
|
||||||
|
|
||||||
for farm in wind_farms:
|
|
||||||
is_enabled = farm.get('enabled', True)
|
|
||||||
if is_enabled:
|
|
||||||
enabled.append(farm)
|
|
||||||
else:
|
|
||||||
disabled.append(farm)
|
|
||||||
|
|
||||||
return enabled, disabled
|
|
||||||
|
|
||||||
|
|
||||||
def get_location_bounds(farm: dict, turbine_df: pd.DataFrame, api_fetcher: APIDataFetcher) -> dict:
|
|
||||||
"""Get location bounds for API query."""
|
|
||||||
location_config = farm['api_params']['location_bounds']
|
|
||||||
|
|
||||||
if location_config['method'] == 'auto':
|
|
||||||
max_distance_ring = max(farm['distance_rings'])
|
|
||||||
padding_km = location_config.get('padding_km', 5)
|
|
||||||
|
|
||||||
bounds = api_fetcher.calculate_location_bounds(
|
|
||||||
turbine_df,
|
|
||||||
max_distance_ring,
|
|
||||||
padding_km
|
|
||||||
)
|
|
||||||
return bounds
|
|
||||||
else:
|
|
||||||
return {
|
|
||||||
'center_lat': location_config['center_lat'],
|
|
||||||
'center_lng': location_config['center_lng'],
|
|
||||||
'radius_km': location_config['radius_km']
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def update_global_config(farm: dict, start_date: str = None, end_date: str = None):
|
|
||||||
"""Update global config with farm-specific settings."""
|
|
||||||
global_config.distance_rings = farm.get('distance_rings', global_config.distance_rings)
|
|
||||||
global_config.ring_colors = farm.get('ring_colors', global_config.ring_colors)
|
|
||||||
# DOCX title is based on the top-level `name` field for the farm.
|
|
||||||
global_config.wind_farm_name = farm.get('name', 'Unknown')
|
|
||||||
global_config.timezone = farm['report_config'].get('timezone', None)
|
|
||||||
|
|
||||||
# Lightning data source configuration (auto-detected from farm config)
|
|
||||||
lightning_source_type = farm.get('lightning_source_type')
|
|
||||||
if lightning_source_type:
|
|
||||||
global_config.lightning_source_type = lightning_source_type
|
|
||||||
if lightning_source_type == 'csv':
|
|
||||||
global_config.lightning_csv = farm.get('lightning_csv')
|
|
||||||
elif lightning_source_type == 'api':
|
|
||||||
global_config.lightning_json = farm.get('lightning_json')
|
|
||||||
|
|
||||||
# Set date range if provided (for reporting)
|
|
||||||
if start_date and end_date:
|
|
||||||
global_config.analysis_start_date = start_date
|
|
||||||
global_config.analysis_end_date = end_date
|
|
||||||
|
|
||||||
# Update grouping params if specified in farm config
|
|
||||||
if 'grouping_params' in farm:
|
|
||||||
global_config.grouping_params = farm['grouping_params']
|
|
||||||
|
|
||||||
logger.debug(f"Updated global config: distance_rings={global_config.distance_rings}, wind_farm_name={global_config.wind_farm_name}")
|
|
||||||
|
|
||||||
|
|
||||||
def convert_api_response_to_dataframe(records: list, data_type: str = 'lightning') -> pd.DataFrame:
|
|
||||||
"""
|
|
||||||
Convert API response to DataFrame format expected by existing code.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
records: List of records from API
|
|
||||||
data_type: 'lightning' or 'storm'
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
DataFrame in expected format
|
|
||||||
"""
|
|
||||||
if not records:
|
|
||||||
if data_type == 'lightning':
|
|
||||||
return pd.DataFrame(columns=['lat', 'lng', 'current', 'p_type', 'local_time'])
|
|
||||||
else:
|
|
||||||
return pd.DataFrame()
|
|
||||||
|
|
||||||
df = pd.DataFrame(records)
|
|
||||||
|
|
||||||
if data_type == 'lightning':
|
|
||||||
if 'local_time' not in df.columns and 'timestamp' in df.columns:
|
|
||||||
df['local_time'] = pd.to_datetime(df['timestamp'])
|
|
||||||
elif 'local_time' in df.columns:
|
|
||||||
df['local_time'] = pd.to_datetime(df['local_time'])
|
|
||||||
|
|
||||||
if 'current_abs' not in df.columns and 'current' in df.columns:
|
|
||||||
df['current_abs'] = df['current'].abs()
|
|
||||||
|
|
||||||
return df
|
|
||||||
|
|
||||||
|
|
||||||
def process_farm(farm: dict, api_fetcher: APIDataFetcher, config: dict) -> dict:
|
|
||||||
"""Process a single farm and generate report."""
|
|
||||||
farm_id = farm['farm_id']
|
|
||||||
farm_name = farm.get('name', farm_id)
|
|
||||||
|
|
||||||
logger.info(f"Processing farm: {farm_id} ({farm_name})")
|
|
||||||
|
|
||||||
try:
|
|
||||||
start_time = datetime.now()
|
|
||||||
|
|
||||||
# Update global config with farm-specific settings BEFORE processing
|
|
||||||
# (dates will be set later after they're determined)
|
|
||||||
update_global_config(farm)
|
|
||||||
|
|
||||||
turbine_file = farm['coordinates_file']
|
|
||||||
turbine_df = load_turbine_data(turbine_file)
|
|
||||||
logger.info(f"Loaded {len(turbine_df)} turbines")
|
|
||||||
|
|
||||||
location_bounds = get_location_bounds(farm, turbine_df, api_fetcher)
|
|
||||||
|
|
||||||
query_start, query_end = APIDataFetcher.determine_query_date_range(farm, config['api_config'])
|
|
||||||
start_date_str = query_start.strftime('%Y-%m-%d')
|
|
||||||
end_date_str = query_end.strftime('%Y-%m-%d')
|
|
||||||
|
|
||||||
source_type = farm.get('lightning_source_type', 'api')
|
|
||||||
|
|
||||||
if source_type == 'csv':
|
|
||||||
lightning_df = load_lightning_data_from_csv(farm.get('lightning_csv'))
|
|
||||||
logger.info(f"Loaded {len(lightning_df)} lightning records from CSV for {farm_id}")
|
|
||||||
else:
|
|
||||||
logger.info(f"Fetching lightning data from API for period: {start_date_str} to {end_date_str}")
|
|
||||||
lightning_records = api_fetcher.fetch_lightning_data(
|
|
||||||
center_lat=location_bounds['center_lat'],
|
|
||||||
center_lng=location_bounds['center_lng'],
|
|
||||||
radius_km=location_bounds['radius_km'],
|
|
||||||
start_date=start_date_str,
|
|
||||||
end_date=end_date_str
|
|
||||||
)
|
|
||||||
lightning_df = convert_api_response_to_dataframe(lightning_records, 'lightning')
|
|
||||||
logger.info(f"Converted {len(lightning_df)} lightning records to DataFrame")
|
|
||||||
|
|
||||||
if len(lightning_df) == 0:
|
|
||||||
logger.warning(f"No lightning data found for {farm_id}")
|
|
||||||
lightning_df = pd.DataFrame(columns=['lat', 'lng', 'current', 'p_type', 'local_time', 'current_abs'])
|
|
||||||
|
|
||||||
storm_records = api_fetcher.fetch_storm_data(
|
|
||||||
center_lat=location_bounds['center_lat'],
|
|
||||||
center_lng=location_bounds['center_lng'],
|
|
||||||
radius_km=location_bounds['radius_km'],
|
|
||||||
start_date=start_date_str,
|
|
||||||
end_date=end_date_str
|
|
||||||
)
|
|
||||||
|
|
||||||
date_range_cfg = farm.get('api_params', {}).get('date_range', {})
|
|
||||||
start_filter = None
|
|
||||||
end_filter = None
|
|
||||||
method = date_range_cfg.get('method')
|
|
||||||
|
|
||||||
if source_type != 'csv':
|
|
||||||
if method == 'manual':
|
|
||||||
start_filter = date_range_cfg.get('start_date')
|
|
||||||
end_filter = date_range_cfg.get('end_date')
|
|
||||||
else:
|
|
||||||
query_range_cfg = date_range_cfg.get('query_range', {})
|
|
||||||
start_filter = query_range_cfg.get('start_date')
|
|
||||||
end_filter = query_range_cfg.get('end_date')
|
|
||||||
|
|
||||||
if len(lightning_df) > 0 and (start_filter is not None or end_filter is not None):
|
|
||||||
lightning_df = filter_lightning_data_by_date_range(lightning_df, start_filter, end_filter)
|
|
||||||
|
|
||||||
farm_tz = farm.get('report_config', {}).get('timezone')
|
|
||||||
if len(lightning_df) > 0 and farm_tz:
|
|
||||||
lightning_df = normalize_local_time_to_timezone(lightning_df, 'local_time', farm_tz)
|
|
||||||
|
|
||||||
turbine_df = calculate_turbine_risks(turbine_df, lightning_df)
|
|
||||||
|
|
||||||
group_data = create_turbine_groups(turbine_df)
|
|
||||||
logger.info(f"Created {group_data['total_groups']} groups")
|
|
||||||
|
|
||||||
# Determine actual dates for report (display strings: DD-MM-YYYY or DD-MM-YYYY HH:MM in local time)
|
|
||||||
if source_type == 'csv' and len(lightning_df) > 0:
|
|
||||||
local_times = pd.to_datetime(lightning_df['local_time'])
|
|
||||||
start_val = local_times.min()
|
|
||||||
end_val = local_times.max()
|
|
||||||
actual_start = start_val.strftime('%d-%m-%Y %H:%M')
|
|
||||||
actual_end = end_val.strftime('%d-%m-%Y %H:%M')
|
|
||||||
else:
|
|
||||||
if method == 'manual' and date_range_cfg.get('start_date') is not None and date_range_cfg.get('end_date') is not None:
|
|
||||||
actual_start, actual_end = format_period_display_for_report(start_filter, end_filter)
|
|
||||||
if not actual_start or not actual_end:
|
|
||||||
actual_start = format_date_ddmmyyyy(query_start)
|
|
||||||
actual_end = format_date_ddmmyyyy(query_end)
|
|
||||||
elif start_filter is not None and end_filter is not None:
|
|
||||||
actual_start, actual_end = format_period_display_for_report(start_filter, end_filter)
|
|
||||||
if not actual_start or not actual_end:
|
|
||||||
actual_start = format_date_ddmmyyyy(query_start)
|
|
||||||
actual_end = format_date_ddmmyyyy(query_end)
|
|
||||||
else:
|
|
||||||
actual_start = format_date_ddmmyyyy(query_start)
|
|
||||||
actual_end = format_date_ddmmyyyy(query_end)
|
|
||||||
|
|
||||||
# Update global config with dates for PDF generation
|
|
||||||
update_global_config(farm, actual_start, actual_end)
|
|
||||||
|
|
||||||
output_dir = Path(farm['report_config']['output_directory'])
|
|
||||||
output_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
local_range = farm_local_date_range_from_config(farm)
|
|
||||||
safe_name = slugify_ascii_underscore(farm.get("name", farm_id))
|
|
||||||
docx_filename = (
|
|
||||||
f"{safe_name}_{local_range.start_date_yyyy_mm_dd}"
|
|
||||||
f"_{local_range.end_date_yyyy_mm_dd}_report.docx"
|
|
||||||
)
|
|
||||||
docx_path = output_dir / docx_filename
|
|
||||||
|
|
||||||
create_docx_report(
|
|
||||||
str(docx_path),
|
|
||||||
turbine_df,
|
|
||||||
lightning_df,
|
|
||||||
storm_data_path=None,
|
|
||||||
storm_data_records=storm_records if storm_records else None,
|
|
||||||
)
|
|
||||||
|
|
||||||
processing_time = (datetime.now() - start_time).total_seconds()
|
|
||||||
|
|
||||||
logger.info(f"Successfully generated report for {farm_id} in {processing_time:.1f}s")
|
|
||||||
|
|
||||||
return {
|
|
||||||
'farm_id': farm_id,
|
|
||||||
'name': farm_name,
|
|
||||||
'status': 'success',
|
|
||||||
'report_path': str(docx_path),
|
|
||||||
'docx_path': str(docx_path),
|
|
||||||
'pdf_path': None,
|
|
||||||
'location': location_bounds,
|
|
||||||
'processing_time_seconds': processing_time,
|
|
||||||
'lightning_records': len(lightning_df),
|
|
||||||
'storm_records': len(storm_records)
|
|
||||||
}
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
processing_time = (datetime.now() - start_time).total_seconds() if 'start_time' in locals() else 0
|
|
||||||
logger.error(f"Failed to process farm {farm_id}: {e}", exc_info=True)
|
|
||||||
|
|
||||||
return {
|
|
||||||
'farm_id': farm_id,
|
|
||||||
'name': farm_name,
|
|
||||||
'status': 'failed',
|
|
||||||
'error': str(e),
|
|
||||||
'processing_time_seconds': processing_time
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def generate_batch_summary(results: list, total_farms: int, enabled_count: int,
|
|
||||||
disabled_count: int, start_time: datetime) -> dict:
|
|
||||||
"""Generate batch summary report."""
|
|
||||||
successful = [r for r in results if r['status'] == 'success']
|
|
||||||
failed = [r for r in results if r['status'] == 'failed']
|
|
||||||
skipped = disabled_count
|
|
||||||
|
|
||||||
total_time = (datetime.now() - start_time).total_seconds()
|
|
||||||
|
|
||||||
summary = {
|
|
||||||
'batch_date': datetime.now().strftime('%Y-%m-%d'),
|
|
||||||
'batch_time': datetime.now().strftime('%H:%M:%S'),
|
|
||||||
'total_farms': total_farms,
|
|
||||||
'enabled_farms': enabled_count,
|
|
||||||
'disabled_farms': disabled_count,
|
|
||||||
'processed': len(results),
|
|
||||||
'successful': len(successful),
|
|
||||||
'failed': len(failed),
|
|
||||||
'skipped': skipped,
|
|
||||||
'processing_time_seconds': total_time,
|
|
||||||
'results': results
|
|
||||||
}
|
|
||||||
|
|
||||||
return summary
|
|
||||||
|
|
||||||
|
|
||||||
def save_batch_summary(summary: dict, output_dir: str):
|
|
||||||
"""Save batch summary to JSON file."""
|
|
||||||
output_path = Path(output_dir) / f"batch_summary_{summary['batch_date']}.json"
|
|
||||||
output_path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
with open(output_path, 'w', encoding='utf-8') as f:
|
|
||||||
json.dump(summary, f, indent=2)
|
|
||||||
|
|
||||||
logger.info(f"Batch summary saved to {output_path}")
|
|
||||||
return output_path
|
|
||||||
|
|
||||||
|
|
||||||
def list_farms(config: dict):
|
|
||||||
"""List all farms and their enabled status."""
|
|
||||||
print("\nWind Farms Configuration:")
|
|
||||||
print("=" * 60)
|
|
||||||
|
|
||||||
for i, farm in enumerate(config['wind_farms'], 1):
|
|
||||||
enabled = farm.get('enabled', True)
|
|
||||||
status = "✓ Enabled" if enabled else "✗ Disabled"
|
|
||||||
farm_id = farm['farm_id']
|
|
||||||
name = farm.get('name', 'N/A')
|
|
||||||
|
|
||||||
print(f"{i}. {status}: {farm_id} - {name}")
|
|
||||||
|
|
||||||
enabled, disabled = filter_enabled_farms(config['wind_farms'])
|
|
||||||
print(f"\nTotal: {len(config['wind_farms'])} farms ({len(enabled)} enabled, {len(disabled)} disabled)")
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
parser = argparse.ArgumentParser(description='Batch generate lightning reports')
|
|
||||||
parser.add_argument('--config', required=True, help='Path to wind_farms_config.json')
|
|
||||||
parser.add_argument('--farm-id', help='Process specific farm only')
|
|
||||||
parser.add_argument('--force-all', action='store_true',
|
|
||||||
help='Process all farms, ignoring enabled flag')
|
|
||||||
parser.add_argument('--force', action='store_true',
|
|
||||||
help='Process even if disabled')
|
|
||||||
parser.add_argument('--list-farms', action='store_true',
|
|
||||||
help='List all farms and their enabled status')
|
|
||||||
parser.add_argument('--output-dir', help='Override output directory')
|
|
||||||
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
try:
|
|
||||||
config = load_wind_farms_config(args.config)
|
|
||||||
|
|
||||||
if args.list_farms:
|
|
||||||
list_farms(config)
|
|
||||||
return
|
|
||||||
|
|
||||||
api_config = config['api_config']
|
|
||||||
api_fetcher = APIDataFetcher(
|
|
||||||
base_url=api_config['base_url'],
|
|
||||||
timeout=api_config.get('timeout_seconds', 30),
|
|
||||||
retry_attempts=api_config.get('retry_attempts', 3)
|
|
||||||
)
|
|
||||||
|
|
||||||
wind_farms = config['wind_farms']
|
|
||||||
|
|
||||||
if args.force_all:
|
|
||||||
farms_to_process = wind_farms
|
|
||||||
logger.info(f"Processing all {len(farms_to_process)} farms (--force-all)")
|
|
||||||
else:
|
|
||||||
enabled_farms, disabled_farms = filter_enabled_farms(wind_farms)
|
|
||||||
farms_to_process = enabled_farms
|
|
||||||
|
|
||||||
if disabled_farms:
|
|
||||||
logger.info(f"Skipping {len(disabled_farms)} disabled farms:")
|
|
||||||
for farm in disabled_farms:
|
|
||||||
logger.info(f" - {farm['farm_id']}: {farm.get('name', 'N/A')}")
|
|
||||||
|
|
||||||
if args.farm_id:
|
|
||||||
farms_to_process = [f for f in farms_to_process if f['farm_id'] == args.farm_id]
|
|
||||||
if not farms_to_process:
|
|
||||||
logger.error(f"Farm '{args.farm_id}' not found or not enabled")
|
|
||||||
return
|
|
||||||
|
|
||||||
if not farms_to_process:
|
|
||||||
logger.warning("No farms to process")
|
|
||||||
return
|
|
||||||
|
|
||||||
logger.info(f"Processing {len(farms_to_process)} farm(s)")
|
|
||||||
start_time = datetime.now()
|
|
||||||
|
|
||||||
results = []
|
|
||||||
for i, farm in enumerate(farms_to_process, 1):
|
|
||||||
logger.info(f"\n[{i}/{len(farms_to_process)}] Processing {farm['farm_id']}...")
|
|
||||||
result = process_farm(farm, api_fetcher, config)
|
|
||||||
results.append(result)
|
|
||||||
|
|
||||||
enabled_count, disabled_count = filter_enabled_farms(wind_farms)
|
|
||||||
summary = generate_batch_summary(
|
|
||||||
results,
|
|
||||||
len(wind_farms),
|
|
||||||
len(enabled_count),
|
|
||||||
len(disabled_count),
|
|
||||||
start_time
|
|
||||||
)
|
|
||||||
|
|
||||||
output_base = config.get('output_base_directory', 'reports/')
|
|
||||||
save_batch_summary(summary, output_base)
|
|
||||||
|
|
||||||
print("\n" + "=" * 60)
|
|
||||||
print("Batch Processing Summary")
|
|
||||||
print("=" * 60)
|
|
||||||
print(f"Total farms: {summary['total_farms']}")
|
|
||||||
print(f"Enabled: {summary['enabled_farms']}")
|
|
||||||
print(f"Disabled: {summary['disabled_farms']}")
|
|
||||||
print(f"Processed: {summary['processed']}")
|
|
||||||
print(f"Successful: {summary['successful']}")
|
|
||||||
print(f"Failed: {summary['failed']}")
|
|
||||||
print(f"Total time: {summary['processing_time_seconds']:.1f}s")
|
|
||||||
print("=" * 60)
|
|
||||||
|
|
||||||
if summary['failed'] > 0:
|
|
||||||
print("\nFailed farms:")
|
|
||||||
for result in [r for r in results if r['status'] == 'failed']:
|
|
||||||
print(f" - {result['farm_id']}: {result.get('error', 'Unknown error')}")
|
|
||||||
|
|
||||||
except KeyboardInterrupt:
|
|
||||||
logger.info("Batch processing interrupted by user")
|
|
||||||
sys.exit(1)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Batch processing failed: {e}", exc_info=True)
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
|
|
||||||
84
main.py
84
main.py
@ -1,84 +0,0 @@
|
|||||||
import sys
|
|
||||||
import argparse
|
|
||||||
import logging
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
from src.api.data_fetcher import APIDataFetcher
|
|
||||||
|
|
||||||
import batch_generate
|
|
||||||
|
|
||||||
# Set up logging
|
|
||||||
logging.basicConfig(
|
|
||||||
level=logging.INFO,
|
|
||||||
format='%(asctime)s - %(name)s - %(levelname)s - %(message)s',
|
|
||||||
handlers=[
|
|
||||||
logging.StreamHandler(sys.stdout),
|
|
||||||
logging.FileHandler('lightning_report.log')
|
|
||||||
]
|
|
||||||
)
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
def main():
|
|
||||||
"""Generate DOCX reports (from wind_farms_config.json)."""
|
|
||||||
try:
|
|
||||||
parser = argparse.ArgumentParser(description="DOCX lightning report generation (from wind_farms_config.json)")
|
|
||||||
parser.add_argument("--config", default="wind_farms_config.json", help="Path to wind_farms_config.json")
|
|
||||||
parser.add_argument("--farm-id", default=None, help="farm_id to process (if omitted, process enabled farms)")
|
|
||||||
parser.add_argument("--force", action="store_true", help="Process even if farm is disabled")
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
logger.info("Starting DOCX report generation...")
|
|
||||||
config = batch_generate.load_wind_farms_config(args.config)
|
|
||||||
farms = config.get("wind_farms", [])
|
|
||||||
|
|
||||||
api_cfg = config["api_config"]
|
|
||||||
api_fetcher = APIDataFetcher(
|
|
||||||
base_url=api_cfg["base_url"],
|
|
||||||
timeout=api_cfg.get("timeout_seconds", 30),
|
|
||||||
retry_attempts=api_cfg.get("retry_attempts", 3),
|
|
||||||
)
|
|
||||||
|
|
||||||
if args.farm_id:
|
|
||||||
farms_to_process = [f for f in farms if f.get("farm_id") == args.farm_id]
|
|
||||||
if not farms_to_process:
|
|
||||||
logger.error(f"Farm '{args.farm_id}' not found in {args.config}")
|
|
||||||
sys.exit(1)
|
|
||||||
else:
|
|
||||||
farms_to_process = farms
|
|
||||||
|
|
||||||
results: list[dict[str, Any]] = []
|
|
||||||
for idx, farm in enumerate(farms_to_process, 1):
|
|
||||||
farm_id = farm.get("farm_id")
|
|
||||||
name = farm.get("name", farm_id)
|
|
||||||
if not args.force and not farm.get("enabled", True):
|
|
||||||
logger.info(f"[{idx}/{len(farms_to_process)}] Skipping disabled farm: {farm_id} ({name})")
|
|
||||||
continue
|
|
||||||
|
|
||||||
logger.info(f"[{idx}/{len(farms_to_process)}] Processing farm: {farm_id} ({name})")
|
|
||||||
result = batch_generate.process_farm(farm, api_fetcher, config)
|
|
||||||
results.append(result)
|
|
||||||
|
|
||||||
if result.get("status") != "success":
|
|
||||||
logger.error(f"Report generation failed for {farm_id}: {result.get('error')}")
|
|
||||||
# Keep going if doing batch; stop early for single farm.
|
|
||||||
if args.farm_id:
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
for r in results:
|
|
||||||
if r.get("status") == "success":
|
|
||||||
logger.info(f"✅ DOCX report saved as {r.get('docx_path')}")
|
|
||||||
|
|
||||||
logger.info("Lightning report generation completed successfully!")
|
|
||||||
|
|
||||||
except FileNotFoundError as e:
|
|
||||||
logger.error(f"File not found: {e}")
|
|
||||||
sys.exit(1)
|
|
||||||
except ValueError as e:
|
|
||||||
logger.error(f"Data validation error: {e}")
|
|
||||||
sys.exit(1)
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Unexpected error: {e}")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
if __name__ == '__main__':
|
|
||||||
main()
|
|
||||||
Binary file not shown.
@ -1,97 +0,0 @@
|
|||||||
# Lightning Report Service
|
|
||||||
|
|
||||||
Tiny FastAPI wrapper around `create_docx_report()`. Lets the n8n workflow produce reports that are **byte-identical** to the CLI (`batch_generate.py`) output, since it uses the same code path.
|
|
||||||
|
|
||||||
## Why this exists
|
|
||||||
|
|
||||||
n8n 2.x's self-hosted Python Code node runs in a sandbox that strips common builtins (`hasattr`, `getattr`, etc.) and restricts imports, making it impossible to host the 3,500+ line reporting pipeline in a node. This service runs outside the sandbox and is called over HTTP.
|
|
||||||
|
|
||||||
## Endpoints
|
|
||||||
|
|
||||||
| Method | Path | Purpose |
|
|
||||||
|---|---|---|
|
|
||||||
| `GET` | `/health` | Liveness probe |
|
|
||||||
| `POST` | `/generate` | Accepts the `Report: Build Payload` JSON, returns a DOCX binary |
|
|
||||||
|
|
||||||
### Request body for `/generate`
|
|
||||||
|
|
||||||
Mirrors the `Report: Build Payload` Set node. Required keys:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"customer_name": "Example Wind Farm",
|
|
||||||
"timezone": "Europe/Istanbul",
|
|
||||||
"centroid_lat": 40.5,
|
|
||||||
"centroid_lon": 29.7,
|
|
||||||
"boundary_m": 20000,
|
|
||||||
"rings": { "r1": 2000, "r2": 4000, "r3": 6000, "r4": 8000 },
|
|
||||||
"ring_colors": ["#B71C1C", "#F94144", "#F8961E", "#90BE6D"],
|
|
||||||
"t_start": 1729000000000,
|
|
||||||
"t_end": 1729010000000,
|
|
||||||
"n_strikes": 1234,
|
|
||||||
"strikes": [ { "latitude": ..., "longitude": ..., "peakCurrent": ..., "type": "0", "captured": "2026-04-22T13:00:00Z" } ],
|
|
||||||
"turbines": [ { "name": "T1", "latitude": ..., "longitude": ..., "unit_power_mwm": 3.6, ... } ],
|
|
||||||
|
|
||||||
"gemini_text": "optional — if provided, used verbatim; else the service calls Gemini or falls back",
|
|
||||||
"storm_records": [ { "cell_polygon_wkt": "POLYGON(...)", "lightning_severity": "medium", "effective_time": "...", "expire_time": "..." } ]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
The adapter is forgiving about column names: it accepts `lat`/`latitude`, `lng`/`longitude`/`lon`, `current`/`peakCurrent`/`peak_current`, `p_type`/`type`, `local_time`/`captured`/`timestamp`.
|
|
||||||
|
|
||||||
### Response
|
|
||||||
|
|
||||||
- `Content-Type: application/vnd.openxmlformats-officedocument.wordprocessingml.document`
|
|
||||||
- `Content-Disposition: attachment; filename="<slug>_<start>_<end>_report.docx"`
|
|
||||||
- Headers: `X-Report-Filename`, `X-Report-Customer`, `X-Report-Strikes`
|
|
||||||
|
|
||||||
## Running locally
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd lightning_report
|
|
||||||
python -m pip install -r requirements.txt -r report_service/requirements.txt
|
|
||||||
uvicorn report_service.main:app --host 0.0.0.0 --port 8000
|
|
||||||
```
|
|
||||||
|
|
||||||
Sanity check:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -sS http://127.0.0.1:8000/health
|
|
||||||
```
|
|
||||||
|
|
||||||
## Running with Docker (alongside n8n)
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cd lightning_report
|
|
||||||
docker compose -f report_service/docker-compose.yml up --build -d
|
|
||||||
docker logs -f lightning-report-service
|
|
||||||
```
|
|
||||||
|
|
||||||
The compose file attaches the service to an external network named `n8n`. Check your actual network name with `docker network ls` and adjust the `name:` field if needed. Once attached, the n8n container can reach the service at `http://report-service:8000`.
|
|
||||||
|
|
||||||
## Environment variables
|
|
||||||
|
|
||||||
| Variable | Default | Purpose |
|
|
||||||
|---|---|---|
|
|
||||||
| `LOG_LEVEL` | `INFO` | Uvicorn/service log level |
|
|
||||||
| `GEMINI_API_KEY` | _(unset)_ | Only used if n8n doesn't send `gemini_text` in the payload. If unset, the service falls back to the deterministic commentary in `src/reporting/gemini_commentary.py` |
|
|
||||||
| `GEMINI_MODEL` | `gemini-1.5-flash` | Only used when the service calls Gemini itself |
|
|
||||||
|
|
||||||
## n8n configuration
|
|
||||||
|
|
||||||
In the n8n workflow, the `Report: Generate DOCX` HTTP Request node points at:
|
|
||||||
|
|
||||||
```
|
|
||||||
={{ $env.REPORT_SERVICE_URL || 'http://report-service:8000' }}/generate
|
|
||||||
```
|
|
||||||
|
|
||||||
- If n8n and the service share a Docker network, the default hostname works.
|
|
||||||
- Otherwise, set `REPORT_SERVICE_URL` as an n8n environment variable (e.g. `http://192.168.1.10:8000`).
|
|
||||||
|
|
||||||
Response format is set to `file` with output property `report`, which the downstream Slack node uploads directly.
|
|
||||||
|
|
||||||
## Gemini commentary handoff
|
|
||||||
|
|
||||||
The existing n8n branch already has a `Report: Gemini Commentary` HTTP Request node that calls Gemini. Its text is forwarded to this service as `gemini_text`. When present, the service skips its own Gemini call and plugs the text straight into `create_docx_report()`'s commentary slot via a scoped monkey-patch on `generate_gemini_paragraph`.
|
|
||||||
|
|
||||||
If you'd rather let the service handle Gemini end-to-end, you can delete the `Report: Gemini Commentary` node from the n8n workflow and point `Report: Build Payload` directly at `Report: Fetch Thunderstorms`.
|
|
||||||
@ -143,12 +143,19 @@ def _epoch_ms_to_local_str(epoch_ms: Any, timezone_name: str | None) -> str | No
|
|||||||
return None
|
return None
|
||||||
|
|
||||||
|
|
||||||
|
def _coerce_float(value: Any) -> float | None:
|
||||||
|
if value is None or value == "":
|
||||||
|
return None
|
||||||
|
try:
|
||||||
|
return float(value)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return None
|
||||||
|
|
||||||
|
|
||||||
def apply_farm_config(payload: dict[str, Any]) -> None:
|
def apply_farm_config(payload: dict[str, Any]) -> None:
|
||||||
"""
|
"""
|
||||||
Mutate the global `src.config.config` singleton per request.
|
Mutate the global `src.config.config` singleton per request so the rest of
|
||||||
|
the reporting pipeline reads farm-specific values from the n8n payload.
|
||||||
Mirrors what `batch_generate.update_global_config()` does, so the rest of
|
|
||||||
the reporting code can read farm-specific values exactly like it does in CLI mode.
|
|
||||||
"""
|
"""
|
||||||
rings_obj = payload.get("rings") or {}
|
rings_obj = payload.get("rings") or {}
|
||||||
ordered_rings = [int(rings_obj[k]) for k in ("r1", "r2", "r3", "r4", "r5") if k in rings_obj]
|
ordered_rings = [int(rings_obj[k]) for k in ("r1", "r2", "r3", "r4", "r5") if k in rings_obj]
|
||||||
@ -162,6 +169,21 @@ def apply_farm_config(payload: dict[str, Any]) -> None:
|
|||||||
config.wind_farm_name = payload.get("customer_name") or config.wind_farm_name or "Wind Farm"
|
config.wind_farm_name = payload.get("customer_name") or config.wind_farm_name or "Wind Farm"
|
||||||
config.timezone = payload.get("timezone") or config.timezone
|
config.timezone = payload.get("timezone") or config.timezone
|
||||||
|
|
||||||
|
# Trust the farm-wide centroid provided by the n8n workflow instead of
|
||||||
|
# recomputing it from turbine coordinates downstream.
|
||||||
|
centroid_lat = _coerce_float(payload.get("centroid_lat"))
|
||||||
|
centroid_lon = _coerce_float(
|
||||||
|
payload.get("centroid_lon")
|
||||||
|
if payload.get("centroid_lon") is not None
|
||||||
|
else payload.get("centroid_lng")
|
||||||
|
)
|
||||||
|
config.centroid_lat = centroid_lat
|
||||||
|
config.centroid_lon = centroid_lon
|
||||||
|
|
||||||
|
# n8n-supplied monitoring boundary is the authoritative outer analysis radius.
|
||||||
|
boundary_m = _coerce_float(payload.get("boundary_m"))
|
||||||
|
config.analysis_boundary_m = int(boundary_m) if boundary_m is not None else None
|
||||||
|
|
||||||
start_label = _epoch_ms_to_local_str(payload.get("t_start"), config.timezone)
|
start_label = _epoch_ms_to_local_str(payload.get("t_start"), config.timezone)
|
||||||
end_label = _epoch_ms_to_local_str(payload.get("t_end"), config.timezone)
|
end_label = _epoch_ms_to_local_str(payload.get("t_end"), config.timezone)
|
||||||
if start_label:
|
if start_label:
|
||||||
|
|||||||
@ -21,6 +21,9 @@ services:
|
|||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
environment:
|
environment:
|
||||||
- LOG_LEVEL=INFO
|
- LOG_LEVEL=INFO
|
||||||
|
# Shared secret the n8n HTTP Request node must send as `X-Report-Token`.
|
||||||
|
# The service refuses all /generate calls if this is unset.
|
||||||
|
- REPORT_SERVICE_TOKEN=${REPORT_SERVICE_TOKEN:-}
|
||||||
# Only needed if n8n does NOT pre-compute gemini_text and you want the
|
# Only needed if n8n does NOT pre-compute gemini_text and you want the
|
||||||
# service to call Gemini itself.
|
# service to call Gemini itself.
|
||||||
- GEMINI_API_KEY=${GEMINI_API_KEY:-}
|
- GEMINI_API_KEY=${GEMINI_API_KEY:-}
|
||||||
|
|||||||
@ -10,13 +10,14 @@ Run locally:
|
|||||||
"""
|
"""
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import hmac
|
||||||
import logging
|
import logging
|
||||||
import os
|
import os
|
||||||
import tempfile
|
import tempfile
|
||||||
from contextlib import contextmanager
|
from contextlib import contextmanager
|
||||||
from typing import Any
|
from typing import Any
|
||||||
|
|
||||||
from fastapi import FastAPI, HTTPException, Request
|
from fastapi import Depends, FastAPI, Header, HTTPException, Request
|
||||||
from fastapi.responses import JSONResponse, Response
|
from fastapi.responses import JSONResponse, Response
|
||||||
|
|
||||||
from src.reporting import docx as docx_module
|
from src.reporting import docx as docx_module
|
||||||
@ -36,6 +37,21 @@ app = FastAPI(title="Lightning Report Service", version="1.0.0")
|
|||||||
DOCX_MIME = "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
|
DOCX_MIME = "application/vnd.openxmlformats-officedocument.wordprocessingml.document"
|
||||||
|
|
||||||
|
|
||||||
|
def require_report_token(x_report_token: str | None = Header(default=None)) -> None:
|
||||||
|
"""
|
||||||
|
Shared-secret gate for the public `/generate` endpoint.
|
||||||
|
|
||||||
|
Fails closed: if `REPORT_SERVICE_TOKEN` is unset in the environment the
|
||||||
|
service refuses every request rather than silently serving open traffic.
|
||||||
|
"""
|
||||||
|
expected = os.getenv("REPORT_SERVICE_TOKEN")
|
||||||
|
if not expected:
|
||||||
|
logger.error("REPORT_SERVICE_TOKEN is not configured; refusing request")
|
||||||
|
raise HTTPException(status_code=503, detail="Service is not configured")
|
||||||
|
if not x_report_token or not hmac.compare_digest(x_report_token, expected):
|
||||||
|
raise HTTPException(status_code=401, detail="Invalid or missing X-Report-Token")
|
||||||
|
|
||||||
|
|
||||||
@contextmanager
|
@contextmanager
|
||||||
def _override_gemini_commentary(override_text: str | None):
|
def _override_gemini_commentary(override_text: str | None):
|
||||||
"""
|
"""
|
||||||
@ -76,7 +92,7 @@ def health() -> JSONResponse:
|
|||||||
return JSONResponse({"ok": True, "service": "lightning-report", "version": app.version})
|
return JSONResponse({"ok": True, "service": "lightning-report", "version": app.version})
|
||||||
|
|
||||||
|
|
||||||
@app.post("/generate")
|
@app.post("/generate", dependencies=[Depends(require_report_token)])
|
||||||
async def generate(request: Request) -> Response:
|
async def generate(request: Request) -> Response:
|
||||||
try:
|
try:
|
||||||
payload: dict[str, Any] = await request.json()
|
payload: dict[str, Any] = await request.json()
|
||||||
|
|||||||
@ -2,7 +2,6 @@ pandas>=1.5.0
|
|||||||
numpy>=1.21.0
|
numpy>=1.21.0
|
||||||
plotly>=5.15.0
|
plotly>=5.15.0
|
||||||
kaleido>=0.2.1
|
kaleido>=0.2.1
|
||||||
scikit-learn>=1.3.0
|
|
||||||
requests>=2.31.0
|
requests>=2.31.0
|
||||||
python-dotenv>=1.0.0
|
python-dotenv>=1.0.0
|
||||||
python-docx>=1.1.2
|
python-docx>=1.1.2
|
||||||
|
|||||||
@ -1,100 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
|
|
||||||
import json
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
from datetime import datetime
|
|
||||||
from typing import Dict, List, Any
|
|
||||||
import logging
|
|
||||||
from collections import defaultdict
|
|
||||||
|
|
||||||
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
def separate_json_by_month(input_file_path: str, output_dir: str = None) -> Dict[str, str]:
|
|
||||||
"""
|
|
||||||
Separate JSON file into smaller files based on months in creation_time.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
input_file_path: Path to the input JSON file
|
|
||||||
output_dir: Directory to save separated files (optional)
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dictionary mapping month to output file path
|
|
||||||
"""
|
|
||||||
try:
|
|
||||||
logger.info(f"Reading JSON file: {input_file_path}")
|
|
||||||
|
|
||||||
with open(input_file_path, 'r', encoding='utf-8') as f:
|
|
||||||
data = json.load(f)
|
|
||||||
|
|
||||||
logger.info(f"Successfully read {len(data)} records")
|
|
||||||
|
|
||||||
if output_dir is None:
|
|
||||||
input_path = Path(input_file_path)
|
|
||||||
output_dir = input_path.parent / f"{input_path.stem}_separated"
|
|
||||||
|
|
||||||
output_path = Path(output_dir)
|
|
||||||
output_path.mkdir(exist_ok=True)
|
|
||||||
|
|
||||||
month_data = defaultdict(list)
|
|
||||||
|
|
||||||
for record in data:
|
|
||||||
try:
|
|
||||||
creation_time = record['creation_time']
|
|
||||||
date_obj = datetime.strptime(creation_time[:10], '%Y-%m-%d')
|
|
||||||
month_key = date_obj.strftime('%Y-%m')
|
|
||||||
month_data[month_key].append(record)
|
|
||||||
except (KeyError, ValueError) as e:
|
|
||||||
logger.warning(f"Skipping record with invalid creation_time: {e}")
|
|
||||||
continue
|
|
||||||
|
|
||||||
output_files = {}
|
|
||||||
|
|
||||||
for month, records in month_data.items():
|
|
||||||
output_file = output_path / f"firtina_sorgulama_{month}.json"
|
|
||||||
|
|
||||||
logger.info(f"Writing {len(records)} records for {month} to {output_file}")
|
|
||||||
|
|
||||||
with open(output_file, 'w', encoding='utf-8') as f:
|
|
||||||
json.dump(records, f, indent=2, ensure_ascii=False, default=str)
|
|
||||||
|
|
||||||
file_size = output_file.stat().st_size / 1024
|
|
||||||
logger.info(f"Created {output_file} ({file_size:.2f} KB)")
|
|
||||||
output_files[month] = str(output_file)
|
|
||||||
|
|
||||||
logger.info(f"Separation completed. Created {len(output_files)} files in {output_path}")
|
|
||||||
|
|
||||||
return output_files
|
|
||||||
|
|
||||||
except FileNotFoundError:
|
|
||||||
logger.error(f"JSON file not found: {input_file_path}")
|
|
||||||
raise
|
|
||||||
except json.JSONDecodeError as e:
|
|
||||||
logger.error(f"Invalid JSON format: {e}")
|
|
||||||
raise
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Error separating JSON by month: {str(e)}")
|
|
||||||
raise
|
|
||||||
|
|
||||||
def main():
|
|
||||||
"""Main function to handle command line execution."""
|
|
||||||
if len(sys.argv) < 2:
|
|
||||||
print("Usage: python separate_by_month.py <json_file_path> [output_directory]")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
input_file_path = sys.argv[1]
|
|
||||||
output_dir = sys.argv[2] if len(sys.argv) > 2 else None
|
|
||||||
|
|
||||||
try:
|
|
||||||
result_files = separate_json_by_month(input_file_path, output_dir)
|
|
||||||
print(f"Separation completed successfully!")
|
|
||||||
print("Created files:")
|
|
||||||
for month, file_path in result_files.items():
|
|
||||||
print(f" {month}: {file_path}")
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Error: {e}")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@ -1,93 +0,0 @@
|
|||||||
import pandas as pd
|
|
||||||
import numpy as np
|
|
||||||
from typing import List, Tuple, Dict
|
|
||||||
from src.analysis.geospatial import haversine_distance
|
|
||||||
from src.utils import get_grouping_radius_m
|
|
||||||
from src.config import config
|
|
||||||
import logging
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
def group_turbines_by_proximity(turbine_df: pd.DataFrame, max_distance_m: int = None) -> List[List[int]]:
|
|
||||||
"""
|
|
||||||
Group turbines based on proximity within max_distance_m meters.
|
|
||||||
Returns list of groups, where each group is a list of turbine indices.
|
|
||||||
"""
|
|
||||||
if max_distance_m is None:
|
|
||||||
max_distance_m = get_grouping_radius_m()
|
|
||||||
try:
|
|
||||||
from sklearn.cluster import DBSCAN
|
|
||||||
import numpy as np
|
|
||||||
coords = np.radians(turbine_df[['lat', 'lng']].values)
|
|
||||||
eps_rad = (max_distance_m / 1000.0) / 6371.0
|
|
||||||
clustering = DBSCAN(eps=eps_rad, min_samples=1, metric='haversine').fit(coords)
|
|
||||||
labels = clustering.labels_
|
|
||||||
groups = []
|
|
||||||
for label in sorted(set(labels)):
|
|
||||||
idxs = np.where(labels == label)[0].tolist()
|
|
||||||
groups.append(idxs)
|
|
||||||
return groups
|
|
||||||
except Exception:
|
|
||||||
n_turbines = len(turbine_df)
|
|
||||||
groups = []
|
|
||||||
assigned = set()
|
|
||||||
for i in range(n_turbines):
|
|
||||||
if i in assigned:
|
|
||||||
continue
|
|
||||||
current_group = [i]
|
|
||||||
assigned.add(i)
|
|
||||||
for j in range(i + 1, n_turbines):
|
|
||||||
if j in assigned:
|
|
||||||
continue
|
|
||||||
distance = haversine_distance(
|
|
||||||
turbine_df.iloc[i]['lat'], turbine_df.iloc[i]['lng'],
|
|
||||||
turbine_df.iloc[j]['lat'], turbine_df.iloc[j]['lng']
|
|
||||||
)
|
|
||||||
if distance <= max_distance_m:
|
|
||||||
current_group.append(j)
|
|
||||||
assigned.add(j)
|
|
||||||
groups.append(current_group)
|
|
||||||
return groups
|
|
||||||
|
|
||||||
def calculate_group_centroid(turbine_df: pd.DataFrame, group_indices: List[int]) -> Tuple[float, float]:
|
|
||||||
"""
|
|
||||||
Calculate the centroid (center point) of a group of turbines.
|
|
||||||
Returns (lat, lng) of the centroid.
|
|
||||||
"""
|
|
||||||
if not group_indices:
|
|
||||||
return 0.0, 0.0
|
|
||||||
|
|
||||||
lats = turbine_df.iloc[group_indices]['lat'].values
|
|
||||||
lngs = turbine_df.iloc[group_indices]['lng'].values
|
|
||||||
|
|
||||||
centroid_lat = np.mean(lats)
|
|
||||||
centroid_lng = np.mean(lngs)
|
|
||||||
|
|
||||||
return centroid_lat, centroid_lng
|
|
||||||
|
|
||||||
def create_turbine_groups(turbine_df: pd.DataFrame) -> Dict:
|
|
||||||
"""
|
|
||||||
Create turbine groups and calculate their centroids.
|
|
||||||
Returns a dictionary with group information.
|
|
||||||
"""
|
|
||||||
groups = group_turbines_by_proximity(turbine_df)
|
|
||||||
|
|
||||||
group_data = []
|
|
||||||
for i, group_indices in enumerate(groups):
|
|
||||||
centroid_lat, centroid_lng = calculate_group_centroid(turbine_df, group_indices)
|
|
||||||
|
|
||||||
group_info = {
|
|
||||||
'group_id': i,
|
|
||||||
'turbine_indices': group_indices,
|
|
||||||
'centroid_lat': centroid_lat,
|
|
||||||
'centroid_lng': centroid_lng,
|
|
||||||
'turbine_count': len(group_indices),
|
|
||||||
'is_single': len(group_indices) == 1
|
|
||||||
}
|
|
||||||
group_data.append(group_info)
|
|
||||||
|
|
||||||
return {
|
|
||||||
'groups': group_data,
|
|
||||||
'total_groups': len(groups),
|
|
||||||
'total_turbines': len(turbine_df)
|
|
||||||
}
|
|
||||||
@ -1,275 +0,0 @@
|
|||||||
import os
|
|
||||||
import requests
|
|
||||||
import json
|
|
||||||
from datetime import datetime, timedelta
|
|
||||||
from typing import Dict, List, Optional, Tuple
|
|
||||||
import logging
|
|
||||||
from dotenv import load_dotenv
|
|
||||||
import pandas as pd
|
|
||||||
import re
|
|
||||||
|
|
||||||
load_dotenv()
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
class APIDataFetcher:
|
|
||||||
def __init__(self, base_url: str, timeout: int = 30, retry_attempts: int = 3):
|
|
||||||
self.base_url = base_url.rstrip('/')
|
|
||||||
self.api_key = os.getenv('API_KEY')
|
|
||||||
self.timeout = timeout
|
|
||||||
self.retry_attempts = retry_attempts
|
|
||||||
|
|
||||||
if not self.api_key:
|
|
||||||
raise ValueError("API_KEY not found in .env file")
|
|
||||||
|
|
||||||
self.session = requests.Session()
|
|
||||||
self.session.headers.update({
|
|
||||||
'Content-Type': 'application/json',
|
|
||||||
'x-api-key': self.api_key
|
|
||||||
})
|
|
||||||
|
|
||||||
def _make_request(self, endpoint: str, params: Dict) -> Dict:
|
|
||||||
"""Make API request with retry logic."""
|
|
||||||
url = f"{self.base_url}{endpoint}"
|
|
||||||
|
|
||||||
# API key is already in headers (x-api-key)
|
|
||||||
for attempt in range(self.retry_attempts):
|
|
||||||
try:
|
|
||||||
response = self.session.get(url, params=params, timeout=self.timeout)
|
|
||||||
response.raise_for_status()
|
|
||||||
return response.json()
|
|
||||||
except requests.exceptions.Timeout:
|
|
||||||
logger.warning(f"API request timeout (attempt {attempt + 1}/{self.retry_attempts})")
|
|
||||||
if attempt == self.retry_attempts - 1:
|
|
||||||
raise
|
|
||||||
except requests.exceptions.RequestException as e:
|
|
||||||
logger.error(f"API request failed: {e}")
|
|
||||||
if attempt == self.retry_attempts - 1:
|
|
||||||
raise
|
|
||||||
except json.JSONDecodeError as e:
|
|
||||||
logger.error(f"Failed to parse API response: {e}")
|
|
||||||
raise
|
|
||||||
|
|
||||||
raise Exception("API request failed after all retries")
|
|
||||||
|
|
||||||
def fetch_lightning_data(
|
|
||||||
self,
|
|
||||||
center_lat: float,
|
|
||||||
center_lng: float,
|
|
||||||
radius_km: float,
|
|
||||||
start_date: str,
|
|
||||||
end_date: str
|
|
||||||
) -> List[Dict]:
|
|
||||||
"""
|
|
||||||
Fetch lightning data from API.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
center_lat: Center latitude
|
|
||||||
center_lng: Center longitude
|
|
||||||
radius_km: Radius in kilometers (converted to meters for API)
|
|
||||||
start_date: Start date in YYYY-MM-DD format
|
|
||||||
end_date: End date in YYYY-MM-DD format
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
List of lightning records
|
|
||||||
"""
|
|
||||||
endpoint = "/lightning-data/historical/"
|
|
||||||
|
|
||||||
radius_m = int(radius_km * 1000)
|
|
||||||
|
|
||||||
params = {
|
|
||||||
'queryType': 'circle',
|
|
||||||
'centerLongitude': center_lng,
|
|
||||||
'centerLatitude': center_lat,
|
|
||||||
'radius': radius_m,
|
|
||||||
'startDate': start_date,
|
|
||||||
'endDate': end_date
|
|
||||||
}
|
|
||||||
|
|
||||||
logger.info(f"Fetching lightning data: center=({center_lat}, {center_lng}), radius={radius_km}km, dates={start_date} to {end_date}")
|
|
||||||
|
|
||||||
try:
|
|
||||||
data = self._make_request(endpoint, params)
|
|
||||||
|
|
||||||
if isinstance(data, dict) and 'data' in data:
|
|
||||||
records = data['data']
|
|
||||||
elif isinstance(data, list):
|
|
||||||
records = data
|
|
||||||
else:
|
|
||||||
records = []
|
|
||||||
|
|
||||||
logger.info(f"Fetched {len(records)} lightning records")
|
|
||||||
return records
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Failed to fetch lightning data: {e}")
|
|
||||||
raise
|
|
||||||
|
|
||||||
def fetch_storm_data(
|
|
||||||
self,
|
|
||||||
center_lat: float,
|
|
||||||
center_lng: float,
|
|
||||||
radius_km: float,
|
|
||||||
start_date: str,
|
|
||||||
end_date: str
|
|
||||||
) -> List[Dict]:
|
|
||||||
"""
|
|
||||||
Fetch storm data from API.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
center_lat: Center latitude
|
|
||||||
center_lng: Center longitude
|
|
||||||
radius_km: Radius in kilometers (converted to meters for API)
|
|
||||||
start_date: Start date in YYYY-MM-DD format
|
|
||||||
end_date: End date in YYYY-MM-DD format
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
List of storm records
|
|
||||||
"""
|
|
||||||
endpoint = "/storm-data/historical/"
|
|
||||||
|
|
||||||
radius_m = int(radius_km * 1000)
|
|
||||||
|
|
||||||
params = {
|
|
||||||
'queryType': 'circle',
|
|
||||||
'centerLongitude': center_lng,
|
|
||||||
'centerLatitude': center_lat,
|
|
||||||
'radius': radius_m,
|
|
||||||
'startDate': start_date,
|
|
||||||
'endDate': end_date
|
|
||||||
}
|
|
||||||
|
|
||||||
logger.info(f"Fetching storm data: center=({center_lat}, {center_lng}), radius={radius_km}km, dates={start_date} to {end_date}")
|
|
||||||
|
|
||||||
try:
|
|
||||||
data = self._make_request(endpoint, params)
|
|
||||||
|
|
||||||
if isinstance(data, dict) and 'data' in data:
|
|
||||||
records = data['data']
|
|
||||||
elif isinstance(data, list):
|
|
||||||
records = data
|
|
||||||
else:
|
|
||||||
records = []
|
|
||||||
|
|
||||||
logger.info(f"Fetched {len(records)} storm records")
|
|
||||||
return records
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Failed to fetch storm data: {e}")
|
|
||||||
raise
|
|
||||||
|
|
||||||
def calculate_location_bounds(
|
|
||||||
self,
|
|
||||||
turbine_df: pd.DataFrame,
|
|
||||||
max_distance_ring_m: int,
|
|
||||||
padding_km: float = 5
|
|
||||||
) -> Dict[str, float]:
|
|
||||||
"""
|
|
||||||
Auto-calculate center + radius from turbine coordinates.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
turbine_df: DataFrame with 'lat' and 'lng' columns
|
|
||||||
max_distance_ring_m: Maximum distance ring in meters
|
|
||||||
padding_km: Additional padding in kilometers
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Dict with center_lat, center_lng, radius_km
|
|
||||||
"""
|
|
||||||
from src.analysis.geospatial import haversine_distance
|
|
||||||
|
|
||||||
centroid_lat = turbine_df['lat'].mean()
|
|
||||||
centroid_lng = turbine_df['lng'].mean()
|
|
||||||
|
|
||||||
max_distance_from_centroid = 0
|
|
||||||
for _, turbine in turbine_df.iterrows():
|
|
||||||
distance = haversine_distance(
|
|
||||||
centroid_lat, centroid_lng,
|
|
||||||
turbine['lat'], turbine['lng']
|
|
||||||
)
|
|
||||||
max_distance_from_centroid = max(max_distance_from_centroid, distance)
|
|
||||||
|
|
||||||
radius_km = (max_distance_from_centroid / 1000) + \
|
|
||||||
(max_distance_ring_m / 1000) + \
|
|
||||||
padding_km
|
|
||||||
|
|
||||||
return {
|
|
||||||
"center_lat": centroid_lat,
|
|
||||||
"center_lng": centroid_lng,
|
|
||||||
"radius_km": radius_km
|
|
||||||
}
|
|
||||||
|
|
||||||
@staticmethod
|
|
||||||
def determine_query_date_range(
|
|
||||||
farm_config: Dict,
|
|
||||||
api_config: Dict
|
|
||||||
) -> Tuple[datetime, datetime]:
|
|
||||||
"""
|
|
||||||
Determine date range for API query.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
farm_config: Farm configuration dict
|
|
||||||
api_config: API configuration dict
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Tuple of (start_date, end_date) as datetime objects
|
|
||||||
"""
|
|
||||||
date_config = farm_config['api_params']['date_range']
|
|
||||||
|
|
||||||
def _parse_config_date(value: str) -> datetime:
|
|
||||||
value_str = str(value).strip()
|
|
||||||
if not value_str:
|
|
||||||
raise ValueError("Empty date value")
|
|
||||||
|
|
||||||
if re.fullmatch(r"\d{2}-\d{2}-\d{4}", value_str):
|
|
||||||
return datetime.strptime(value_str, '%d-%m-%Y')
|
|
||||||
|
|
||||||
ts = pd.to_datetime(value_str, errors='raise')
|
|
||||||
if isinstance(ts, pd.Timestamp):
|
|
||||||
if ts.tzinfo is not None:
|
|
||||||
ts = ts.tz_convert('UTC').tz_localize(None)
|
|
||||||
return ts.to_pydatetime()
|
|
||||||
|
|
||||||
raise ValueError(f"Unsupported date value: {value_str}")
|
|
||||||
|
|
||||||
if date_config['method'] == 'manual':
|
|
||||||
start_date = _parse_config_date(date_config['start_date'])
|
|
||||||
end_date = _parse_config_date(date_config['end_date'])
|
|
||||||
return start_date, end_date
|
|
||||||
|
|
||||||
today = datetime.now()
|
|
||||||
query_range = date_config.get('query_range', {})
|
|
||||||
method = query_range.get('method', api_config.get('default_query_range', {}).get('method', 'current_month'))
|
|
||||||
|
|
||||||
if method == 'current_month':
|
|
||||||
start_date = today.replace(day=1)
|
|
||||||
end_date = today
|
|
||||||
|
|
||||||
elif method == 'last_month':
|
|
||||||
if today.month == 1:
|
|
||||||
start_date = today.replace(year=today.year-1, month=12, day=1)
|
|
||||||
else:
|
|
||||||
start_date = today.replace(month=today.month-1, day=1)
|
|
||||||
|
|
||||||
if today.month == 1:
|
|
||||||
last_day = (today.replace(year=today.year-1, month=12, day=28) +
|
|
||||||
timedelta(days=4)).replace(day=1) - timedelta(days=1)
|
|
||||||
else:
|
|
||||||
last_day = (today.replace(month=today.month, day=28) +
|
|
||||||
timedelta(days=4)).replace(day=1) - timedelta(days=1)
|
|
||||||
end_date = last_day
|
|
||||||
|
|
||||||
elif method == 'days_back':
|
|
||||||
days = query_range.get('days', 30)
|
|
||||||
start_date = today - timedelta(days=days)
|
|
||||||
end_date = today
|
|
||||||
|
|
||||||
elif method == 'custom':
|
|
||||||
start_date = _parse_config_date(query_range['start_date'])
|
|
||||||
end_date = _parse_config_date(query_range['end_date'])
|
|
||||||
|
|
||||||
else:
|
|
||||||
start_date = today.replace(day=1)
|
|
||||||
end_date = today
|
|
||||||
|
|
||||||
return start_date, end_date
|
|
||||||
|
|
||||||
@ -7,20 +7,22 @@ import os
|
|||||||
class Config:
|
class Config:
|
||||||
"""
|
"""
|
||||||
Centralized configuration for global/default settings.
|
Centralized configuration for global/default settings.
|
||||||
|
|
||||||
IMPORTANT: Farm-specific settings (distance_rings, ring_colors, wind_farm_name,
|
Farm-specific values (distance_rings, ring_colors, wind_farm_name, centroid,
|
||||||
file paths, date ranges) are managed in wind_farms_config.json and set dynamically
|
date range, timezone) are populated per request by
|
||||||
by batch_generate.py. The values below are only fallback defaults for backward
|
`report_service.adapter.apply_farm_config()` from the n8n payload. The
|
||||||
compatibility and should NOT be configured here.
|
defaults below are only fallbacks for direct callers that bypass the service.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
# Farm-specific settings (set by batch processing, DO NOT configure here)
|
|
||||||
distance_rings: List[int] = None
|
distance_rings: List[int] = None
|
||||||
ring_colors: List[str] = None
|
ring_colors: List[str] = None
|
||||||
wind_farm_name: str = None
|
wind_farm_name: str = None
|
||||||
analysis_start_date: Optional[str] = None
|
analysis_start_date: Optional[str] = None
|
||||||
analysis_end_date: Optional[str] = None
|
analysis_end_date: Optional[str] = None
|
||||||
timezone: Optional[str] = None
|
timezone: Optional[str] = None
|
||||||
|
centroid_lat: Optional[float] = None
|
||||||
|
centroid_lon: Optional[float] = None
|
||||||
|
analysis_boundary_m: Optional[int] = None
|
||||||
|
|
||||||
# Lightning data source configuration
|
# Lightning data source configuration
|
||||||
# By default, lightning data is expected from API (JSON output).
|
# By default, lightning data is expected from API (JSON output).
|
||||||
@ -35,9 +37,6 @@ class Config:
|
|||||||
|
|
||||||
# Histogram parameters (global defaults)
|
# Histogram parameters (global defaults)
|
||||||
histogram_params: Dict[str, Any] = None
|
histogram_params: Dict[str, Any] = None
|
||||||
|
|
||||||
# Grouping parameters (global defaults, can be overridden per-farm)
|
|
||||||
grouping_params: Dict[str, Any] = None
|
|
||||||
|
|
||||||
# Centralized histogram layout configuration (global defaults)
|
# Centralized histogram layout configuration (global defaults)
|
||||||
histogram_layout: Dict[str, Any] = None
|
histogram_layout: Dict[str, Any] = None
|
||||||
@ -80,13 +79,7 @@ class Config:
|
|||||||
'image_scale': 2,
|
'image_scale': 2,
|
||||||
'image_engine': 'kaleido'
|
'image_engine': 'kaleido'
|
||||||
}
|
}
|
||||||
|
|
||||||
if self.grouping_params is None:
|
|
||||||
self.grouping_params = {
|
|
||||||
'max_distance_m': None,
|
|
||||||
'distance_ring_index': 4,
|
|
||||||
'min_group_size': 1,
|
|
||||||
'max_group_size': 50
|
|
||||||
}
|
|
||||||
# Global configuration instance
|
# Global configuration instance
|
||||||
config = Config()
|
config = Config()
|
||||||
@ -1,205 +0,0 @@
|
|||||||
import logging
|
|
||||||
from typing import Dict, Any
|
|
||||||
|
|
||||||
import pandas as pd
|
|
||||||
|
|
||||||
from ..config import config
|
|
||||||
from ..utils import (
|
|
||||||
ensure_datetime_column,
|
|
||||||
filter_lightning_data_by_date_range,
|
|
||||||
load_json_data,
|
|
||||||
normalize_local_time_to_timezone,
|
|
||||||
validate_lightning_data,
|
|
||||||
validate_turbine_data,
|
|
||||||
)
|
|
||||||
|
|
||||||
logger = logging.getLogger(__name__)
|
|
||||||
|
|
||||||
|
|
||||||
def _rename_columns_for_lightning(df: pd.DataFrame) -> pd.DataFrame:
|
|
||||||
required = {
|
|
||||||
"lat": ["lat", "latitude"],
|
|
||||||
"lng": ["lng", "lon", "longitude", "long"],
|
|
||||||
"current": ["current", "amplitude", "amp", "peak_current"],
|
|
||||||
"p_type": ["p_type", "ptype", "type", "flash_type"],
|
|
||||||
"local_time": ["local_time", "time", "time_utc", "timestamp", "datetime", "date_time", "localtime"],
|
|
||||||
}
|
|
||||||
|
|
||||||
columns_lower = {col.lower(): col for col in df.columns}
|
|
||||||
rename_map: Dict[str, str] = {}
|
|
||||||
|
|
||||||
for target, candidates in required.items():
|
|
||||||
found_source = None
|
|
||||||
for candidate in candidates:
|
|
||||||
source = columns_lower.get(candidate.lower())
|
|
||||||
if source is not None:
|
|
||||||
found_source = source
|
|
||||||
break
|
|
||||||
if found_source is not None and found_source != target:
|
|
||||||
rename_map[found_source] = target
|
|
||||||
|
|
||||||
if rename_map:
|
|
||||||
df = df.rename(columns=rename_map)
|
|
||||||
|
|
||||||
missing = [col for col in ["lat", "lng", "current", "p_type", "local_time"] if col not in df.columns]
|
|
||||||
if missing:
|
|
||||||
raise ValueError(f"Missing required lightning columns in CSV: {missing}")
|
|
||||||
|
|
||||||
return df
|
|
||||||
|
|
||||||
|
|
||||||
def load_lightning_data_from_csv(csv_path: str | None = None) -> pd.DataFrame:
|
|
||||||
"""
|
|
||||||
Load and validate lightning data from a CSV file.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
csv_path: Path to lightning CSV file. If None, uses config.lightning_csv.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
DataFrame containing lightning data
|
|
||||||
"""
|
|
||||||
if csv_path is None:
|
|
||||||
csv_path = getattr(config, "lightning_csv", None)
|
|
||||||
if csv_path is None:
|
|
||||||
raise ValueError("lightning_csv path must be provided (not available in config)")
|
|
||||||
|
|
||||||
logger.info(f"Loading lightning data from CSV {csv_path}")
|
|
||||||
|
|
||||||
try:
|
|
||||||
df = pd.read_csv(csv_path)
|
|
||||||
|
|
||||||
if len(df) == 0:
|
|
||||||
logger.warning(
|
|
||||||
"No lightning records found in CSV - creating empty DataFrame with required columns",
|
|
||||||
)
|
|
||||||
df = pd.DataFrame(columns=["lat", "lng", "current", "p_type", "local_time"])
|
|
||||||
else:
|
|
||||||
df = _rename_columns_for_lightning(df)
|
|
||||||
|
|
||||||
if not validate_lightning_data(df):
|
|
||||||
raise ValueError("Lightning data validation failed for CSV source")
|
|
||||||
|
|
||||||
df = ensure_datetime_column(df, "local_time")
|
|
||||||
|
|
||||||
if "current_abs" not in df.columns and len(df) > 0:
|
|
||||||
df["current_abs"] = df["current"].abs()
|
|
||||||
elif "current_abs" not in df.columns and len(df) == 0:
|
|
||||||
df["current_abs"] = pd.Series(dtype="float64")
|
|
||||||
|
|
||||||
start_date = getattr(config, "analysis_start_date", None)
|
|
||||||
end_date = getattr(config, "analysis_end_date", None)
|
|
||||||
|
|
||||||
if start_date and end_date:
|
|
||||||
df = filter_lightning_data_by_date_range(df, start_date, end_date)
|
|
||||||
|
|
||||||
farm_tz = getattr(config, "timezone", None)
|
|
||||||
if len(df) > 0 and farm_tz:
|
|
||||||
df = normalize_local_time_to_timezone(df, "local_time", farm_tz)
|
|
||||||
|
|
||||||
logger.info(f"Successfully loaded {len(df)} lightning records from CSV")
|
|
||||||
return df
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Failed to load lightning data from CSV: {e}")
|
|
||||||
raise
|
|
||||||
|
|
||||||
|
|
||||||
def load_lightning_data(json_path: str = None) -> pd.DataFrame:
|
|
||||||
"""
|
|
||||||
Load and validate lightning data from configured source.
|
|
||||||
|
|
||||||
If config.lightning_source_type == \"csv\" and no explicit json_path is provided,
|
|
||||||
data will be loaded from CSV. Otherwise, API/JSON loading is used.
|
|
||||||
"""
|
|
||||||
if json_path is None:
|
|
||||||
source_type = getattr(config, "lightning_source_type", "api")
|
|
||||||
if source_type == "csv":
|
|
||||||
return load_lightning_data_from_csv()
|
|
||||||
|
|
||||||
json_path = getattr(config, "lightning_json", None)
|
|
||||||
if json_path is None:
|
|
||||||
raise ValueError("lightning_json path must be provided (not available in config)")
|
|
||||||
|
|
||||||
logger.info(f"Loading lightning data from JSON {json_path}")
|
|
||||||
|
|
||||||
try:
|
|
||||||
data = load_json_data(json_path)
|
|
||||||
|
|
||||||
if isinstance(data, dict) and len(data) > 0:
|
|
||||||
if "data" in data and isinstance(data["data"], list):
|
|
||||||
records = data["data"]
|
|
||||||
else:
|
|
||||||
records = list(data.values())[0]
|
|
||||||
else:
|
|
||||||
records = data
|
|
||||||
|
|
||||||
df = pd.DataFrame(records)
|
|
||||||
|
|
||||||
if len(df) == 0:
|
|
||||||
logger.warning("No lightning records found in data - creating empty DataFrame with required columns")
|
|
||||||
df = pd.DataFrame(columns=["lat", "lng", "current", "p_type", "local_time"])
|
|
||||||
|
|
||||||
if not validate_lightning_data(df):
|
|
||||||
raise ValueError("Lightning data validation failed")
|
|
||||||
|
|
||||||
df = ensure_datetime_column(df, "local_time")
|
|
||||||
|
|
||||||
if "current_abs" not in df.columns and len(df) > 0:
|
|
||||||
df["current_abs"] = df["current"].abs()
|
|
||||||
elif "current_abs" not in df.columns and len(df) == 0:
|
|
||||||
df["current_abs"] = pd.Series(dtype="float64")
|
|
||||||
|
|
||||||
start_date = getattr(config, "analysis_start_date", None)
|
|
||||||
end_date = getattr(config, "analysis_end_date", None)
|
|
||||||
|
|
||||||
if start_date and end_date:
|
|
||||||
df = filter_lightning_data_by_date_range(df, start_date, end_date)
|
|
||||||
|
|
||||||
farm_tz = getattr(config, "timezone", None)
|
|
||||||
if len(df) > 0 and farm_tz:
|
|
||||||
df = normalize_local_time_to_timezone(df, "local_time", farm_tz)
|
|
||||||
|
|
||||||
logger.info(f"Successfully loaded {len(df)} lightning records")
|
|
||||||
return df
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Failed to load lightning data: {e}")
|
|
||||||
raise
|
|
||||||
|
|
||||||
def load_turbine_data(json_path: str = None) -> pd.DataFrame:
|
|
||||||
"""
|
|
||||||
Load and validate turbine data from JSON file.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
json_path: Path to turbine JSON file. If None, uses config default.
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
DataFrame containing turbine data
|
|
||||||
|
|
||||||
Raises:
|
|
||||||
ValueError: If data validation fails
|
|
||||||
"""
|
|
||||||
if json_path is None:
|
|
||||||
json_path = getattr(config, 'turbine_json', None)
|
|
||||||
if json_path is None:
|
|
||||||
raise ValueError("turbine_json path must be provided (not available in config)")
|
|
||||||
|
|
||||||
logger.info(f"Loading turbine data from {json_path}")
|
|
||||||
|
|
||||||
try:
|
|
||||||
# Load JSON data
|
|
||||||
data = load_json_data(json_path)
|
|
||||||
|
|
||||||
# Create DataFrame
|
|
||||||
df = pd.DataFrame(data)
|
|
||||||
|
|
||||||
# Validate data
|
|
||||||
if not validate_turbine_data(df):
|
|
||||||
raise ValueError("Turbine data validation failed")
|
|
||||||
|
|
||||||
logger.info(f"Successfully loaded {len(df)} turbine records")
|
|
||||||
return df
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
logger.error(f"Failed to load turbine data: {e}")
|
|
||||||
raise
|
|
||||||
@ -15,15 +15,14 @@ from docx.oxml import OxmlElement
|
|||||||
from docx.oxml.ns import qn
|
from docx.oxml.ns import qn
|
||||||
from docx.shared import Cm, Inches, Pt, RGBColor
|
from docx.shared import Cm, Inches, Pt, RGBColor
|
||||||
|
|
||||||
from src.analysis.grouping import create_turbine_groups
|
|
||||||
from src.analysis.histogram import create_lightning_histogram_pages
|
from src.analysis.histogram import create_lightning_histogram_pages
|
||||||
from src.analysis.risk import calculate_turbine_risks
|
from src.analysis.risk import calculate_turbine_risks
|
||||||
from src.analysis.statistics import calculate_lightning_statistics
|
from src.analysis.statistics import calculate_lightning_statistics
|
||||||
from src.config import config
|
from src.config import config
|
||||||
from src.reporting.precompute import precompute_group_distances_and_rings
|
from src.reporting.precompute import precompute_distances_and_rings
|
||||||
from src.reporting.docx_sections import (
|
from src.reporting.docx_sections import (
|
||||||
_build_current_vs_distance_chart,
|
_build_current_vs_distance_chart,
|
||||||
build_group_lightning_table_data,
|
build_lightning_event_table_data,
|
||||||
build_risk_table_data,
|
build_risk_table_data,
|
||||||
)
|
)
|
||||||
from src.utils import (
|
from src.utils import (
|
||||||
@ -461,8 +460,17 @@ def create_docx_report(
|
|||||||
|
|
||||||
_add_paragraph(doc, period_label, size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER, bold=True)
|
_add_paragraph(doc, period_label, size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER, bold=True)
|
||||||
_add_paragraph(doc, f"{start_date or ''} - {end_date or ''}", size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER)
|
_add_paragraph(doc, f"{start_date or ''} - {end_date or ''}", size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER)
|
||||||
centroid_lat = float(turbine_df["lat"].mean()) if len(turbine_df) > 0 else 0.0
|
# Prefer the farm-wide centroid supplied by the n8n payload (via
|
||||||
centroid_lng = float(turbine_df["lng"].mean()) if len(turbine_df) > 0 else 0.0
|
# `report_service.adapter.apply_farm_config`). Fall back to the turbine
|
||||||
|
# mean only when the service did not pre-configure it (e.g. direct callers).
|
||||||
|
if config.centroid_lat is not None:
|
||||||
|
centroid_lat = float(config.centroid_lat)
|
||||||
|
else:
|
||||||
|
centroid_lat = float(turbine_df["lat"].mean()) if len(turbine_df) > 0 else 0.0
|
||||||
|
if config.centroid_lon is not None:
|
||||||
|
centroid_lng = float(config.centroid_lon)
|
||||||
|
else:
|
||||||
|
centroid_lng = float(turbine_df["lng"].mean()) if len(turbine_df) > 0 else 0.0
|
||||||
_add_paragraph(doc, "Centroid Coordinates:", size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER, bold=True)
|
_add_paragraph(doc, "Centroid Coordinates:", size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER, bold=True)
|
||||||
_add_paragraph(doc, f"Latitude: {centroid_lat:.5f}", size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER)
|
_add_paragraph(doc, f"Latitude: {centroid_lat:.5f}", size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER)
|
||||||
_add_paragraph(doc, f"Longitude: {centroid_lng:.5f}", size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER)
|
_add_paragraph(doc, f"Longitude: {centroid_lng:.5f}", size_pt=12, align=WD_ALIGN_PARAGRAPH.CENTER)
|
||||||
@ -499,9 +507,7 @@ def create_docx_report(
|
|||||||
storm_data = filter_storm_data_by_date_range(storm_data, start_date, end_date)
|
storm_data = filter_storm_data_by_date_range(storm_data, start_date, end_date)
|
||||||
storm_data = filter_storm_data_by_turbine_proximity(storm_data, turbine_df)
|
storm_data = filter_storm_data_by_turbine_proximity(storm_data, turbine_df)
|
||||||
|
|
||||||
# Compute risks and groups
|
|
||||||
turbine_df = calculate_turbine_risks(turbine_df, lightning_df)
|
turbine_df = calculate_turbine_risks(turbine_df, lightning_df)
|
||||||
group_data = create_turbine_groups(turbine_df)
|
|
||||||
stats = calculate_lightning_statistics(lightning_df, centroid_lat, centroid_lng, start_date, end_date)
|
stats = calculate_lightning_statistics(lightning_df, centroid_lat, centroid_lng, start_date, end_date)
|
||||||
histogram_figs = create_lightning_histogram_pages(lightning_df, centroid_lat, centroid_lng)
|
histogram_figs = create_lightning_histogram_pages(lightning_df, centroid_lat, centroid_lng)
|
||||||
|
|
||||||
@ -669,67 +675,66 @@ def create_docx_report(
|
|||||||
|
|
||||||
doc.add_page_break()
|
doc.add_page_break()
|
||||||
|
|
||||||
# Group maps + charts + risk tables
|
# Farm-wide maps + charts + risk table, anchored at the n8n-supplied centroid.
|
||||||
for group_info in group_data["groups"]:
|
centroid_row = pd.Series({
|
||||||
centroid_row = pd.Series({
|
"lat": centroid_lat,
|
||||||
"lat": group_info["centroid_lat"],
|
"lng": centroid_lng,
|
||||||
"lng": group_info["centroid_lng"],
|
"name": config.wind_farm_name or "Wind Farm",
|
||||||
"name": f"Group {group_info['group_id'] + 1}",
|
})
|
||||||
})
|
pre = precompute_distances_and_rings(centroid_lat, centroid_lng, lightning_df, config.distance_rings)
|
||||||
pre = precompute_group_distances_and_rings(centroid_row["lat"], centroid_row["lng"], lightning_df, config.distance_rings)
|
|
||||||
|
|
||||||
_add_title(doc, "Cloud-to-Ground Lightnings", size_pt=14)
|
_add_title(doc, "Cloud-to-Ground Lightnings", size_pt=14)
|
||||||
if start_date and end_date:
|
if start_date and end_date:
|
||||||
_add_paragraph(doc, f"{start_date} - {end_date}", size_pt=10)
|
_add_paragraph(doc, f"{start_date} - {end_date}", size_pt=10)
|
||||||
cg_fig = plot_cloud_to_ground_coordinate_plane(centroid_row, lightning_df, turbine_df, storm_data, precomputed=pre)
|
cg_fig = plot_cloud_to_ground_coordinate_plane(centroid_row, lightning_df, turbine_df, storm_data, precomputed=pre)
|
||||||
_add_image_from_bytes(doc, cg_fig.to_image(format="png", width=1200, height=900, scale=2, engine="kaleido"), content_width)
|
_add_image_from_bytes(doc, cg_fig.to_image(format="png", width=1200, height=900, scale=2, engine="kaleido"), content_width)
|
||||||
|
doc.add_page_break()
|
||||||
|
|
||||||
|
_add_title(doc, "Cloud-to-Ground — Current vs Distance", size_pt=14)
|
||||||
|
if start_date and end_date:
|
||||||
|
_add_paragraph(doc, f"{start_date} - {end_date}", size_pt=10)
|
||||||
|
cg_chart = _build_current_vs_distance_chart(
|
||||||
|
lightning_df,
|
||||||
|
pre["dists_km"],
|
||||||
|
pre["mask_within"],
|
||||||
|
"cg",
|
||||||
|
"Cloud-to-Ground Lightning — Current vs Distance from Centroid",
|
||||||
|
900,
|
||||||
|
650,
|
||||||
|
)
|
||||||
|
if cg_chart is not None:
|
||||||
|
_add_image_from_bytes(doc, cg_chart.to_image(format="png", width=1200, height=900, scale=2, engine="kaleido"), content_width)
|
||||||
|
doc.add_page_break()
|
||||||
|
|
||||||
|
_add_title(doc, "Intercloud Lightnings", size_pt=14)
|
||||||
|
if start_date and end_date:
|
||||||
|
_add_paragraph(doc, f"{start_date} - {end_date}", size_pt=10)
|
||||||
|
ic_fig = plot_intercloud_coordinate_plane(centroid_row, lightning_df, turbine_df, storm_data, precomputed=pre)
|
||||||
|
_add_image_from_bytes(doc, ic_fig.to_image(format="png", width=1200, height=900, scale=2, engine="kaleido"), content_width)
|
||||||
|
doc.add_page_break()
|
||||||
|
|
||||||
|
_add_title(doc, "Intercloud — Current vs Distance", size_pt=14)
|
||||||
|
if start_date and end_date:
|
||||||
|
_add_paragraph(doc, f"{start_date} - {end_date}", size_pt=10)
|
||||||
|
ic_chart = _build_current_vs_distance_chart(
|
||||||
|
lightning_df,
|
||||||
|
pre["dists_km"],
|
||||||
|
pre["mask_within"],
|
||||||
|
"ic",
|
||||||
|
"Intercloud Lightning — Current vs Distance from Centroid",
|
||||||
|
900,
|
||||||
|
650,
|
||||||
|
)
|
||||||
|
if ic_chart is not None:
|
||||||
|
_add_image_from_bytes(doc, ic_chart.to_image(format="png", width=1200, height=900, scale=2, engine="kaleido"), content_width)
|
||||||
|
doc.add_page_break()
|
||||||
|
|
||||||
|
risk_table_data, risk_row_colors = build_risk_table_data(turbine_df)
|
||||||
|
if risk_table_data and len(risk_table_data) > 1:
|
||||||
|
_add_title(doc, "Turbine Risk Assessment", size_pt=14)
|
||||||
|
_add_table(doc, risk_table_data, row_colors=risk_row_colors)
|
||||||
doc.add_page_break()
|
doc.add_page_break()
|
||||||
|
|
||||||
_add_title(doc, "Cloud-to-Ground — Current vs Distance", size_pt=14)
|
|
||||||
if start_date and end_date:
|
|
||||||
_add_paragraph(doc, f"{start_date} - {end_date}", size_pt=10)
|
|
||||||
cg_chart = _build_current_vs_distance_chart(
|
|
||||||
lightning_df,
|
|
||||||
pre["dists_km"],
|
|
||||||
pre["mask_within"],
|
|
||||||
"cg",
|
|
||||||
"Cloud-to-Ground Lightning — Current vs Distance from Centroid",
|
|
||||||
900,
|
|
||||||
650,
|
|
||||||
)
|
|
||||||
if cg_chart is not None:
|
|
||||||
_add_image_from_bytes(doc, cg_chart.to_image(format="png", width=1200, height=900, scale=2, engine="kaleido"), content_width)
|
|
||||||
doc.add_page_break()
|
|
||||||
|
|
||||||
_add_title(doc, "Intercloud Lightnings", size_pt=14)
|
|
||||||
if start_date and end_date:
|
|
||||||
_add_paragraph(doc, f"{start_date} - {end_date}", size_pt=10)
|
|
||||||
ic_fig = plot_intercloud_coordinate_plane(centroid_row, lightning_df, turbine_df, storm_data, precomputed=pre)
|
|
||||||
_add_image_from_bytes(doc, ic_fig.to_image(format="png", width=1200, height=900, scale=2, engine="kaleido"), content_width)
|
|
||||||
doc.add_page_break()
|
|
||||||
|
|
||||||
_add_title(doc, "Intercloud — Current vs Distance", size_pt=14)
|
|
||||||
if start_date and end_date:
|
|
||||||
_add_paragraph(doc, f"{start_date} - {end_date}", size_pt=10)
|
|
||||||
ic_chart = _build_current_vs_distance_chart(
|
|
||||||
lightning_df,
|
|
||||||
pre["dists_km"],
|
|
||||||
pre["mask_within"],
|
|
||||||
"ic",
|
|
||||||
"Intercloud Lightning — Current vs Distance from Centroid",
|
|
||||||
900,
|
|
||||||
650,
|
|
||||||
)
|
|
||||||
if ic_chart is not None:
|
|
||||||
_add_image_from_bytes(doc, ic_chart.to_image(format="png", width=1200, height=900, scale=2, engine="kaleido"), content_width)
|
|
||||||
doc.add_page_break()
|
|
||||||
|
|
||||||
risk_table_data, risk_row_colors = build_risk_table_data(turbine_df, group_info)
|
|
||||||
if risk_table_data and len(risk_table_data) > 1:
|
|
||||||
_add_title(doc, "Turbine Risk Assessment", size_pt=14)
|
|
||||||
_add_table(doc, risk_table_data, row_colors=risk_row_colors)
|
|
||||||
doc.add_page_break()
|
|
||||||
|
|
||||||
# Daily breakdown
|
# Daily breakdown
|
||||||
_add_title(doc, "Lightning Breakdown by Distance Rings", size_pt=14)
|
_add_title(doc, "Lightning Breakdown by Distance Rings", size_pt=14)
|
||||||
if start_date and end_date:
|
if start_date and end_date:
|
||||||
@ -897,38 +902,37 @@ def create_docx_report(
|
|||||||
)
|
)
|
||||||
doc.add_page_break()
|
doc.add_page_break()
|
||||||
|
|
||||||
# Lightning event tables per group
|
# Farm-wide lightning event table, anchored at the n8n-supplied centroid.
|
||||||
for group_info in group_data["groups"]:
|
_add_title(doc, "Detailed Lightning Event Data", size_pt=14)
|
||||||
_add_title(doc, "Detailed Lightning Event Data", size_pt=14)
|
table_data, row_colors = build_lightning_event_table_data(centroid_lat, centroid_lng, lightning_df)
|
||||||
table_data, row_colors = build_group_lightning_table_data(group_info["centroid_lat"], group_info["centroid_lng"], lightning_df)
|
if table_data and table_data[0]:
|
||||||
if table_data and table_data[0]:
|
table_data[0] = [
|
||||||
table_data[0] = [
|
str(h)
|
||||||
str(h)
|
.replace("Time (Local)", "Time\u00A0(Local)")
|
||||||
.replace("Time (Local)", "Time\u00A0(Local)")
|
.replace("Current (amps)", "Current\u00A0(amps)")
|
||||||
.replace("Current (amps)", "Current\u00A0(amps)")
|
.replace("Height (m)", "Height\u00A0(m)")
|
||||||
.replace("Height (m)", "Height\u00A0(m)")
|
.replace("Lightning Type", "Lightning\u00A0Type")
|
||||||
.replace("Lightning Type", "Lightning\u00A0Type")
|
.replace("Proximity (km)", "Proximity\u00A0(km)")
|
||||||
.replace("Proximity (km)", "Proximity\u00A0(km)")
|
for h in table_data[0]
|
||||||
for h in table_data[0]
|
]
|
||||||
]
|
available_width_cm = float(content_width) * 2.54
|
||||||
available_width_cm = float(content_width) * 2.54
|
min_col_widths_cm = [1, 3.2, 1.6, 1.6, 1.9, 1.4, 2.0, 1.8]
|
||||||
min_col_widths_cm = [1, 3.2, 1.6, 1.6, 1.9, 1.4, 2.0, 1.8]
|
font_pt, col_widths_cm = _fit_one_line_table_layout(
|
||||||
font_pt, col_widths_cm = _fit_one_line_table_layout(
|
table_data=table_data,
|
||||||
table_data=table_data,
|
available_width_cm=available_width_cm,
|
||||||
available_width_cm=available_width_cm,
|
min_col_widths_cm=min_col_widths_cm,
|
||||||
min_col_widths_cm=min_col_widths_cm,
|
max_font_pt=15,
|
||||||
max_font_pt=15,
|
min_font_pt=8,
|
||||||
min_font_pt=8,
|
)
|
||||||
)
|
_add_table(
|
||||||
_add_table(
|
doc,
|
||||||
doc,
|
table_data,
|
||||||
table_data,
|
row_colors=row_colors,
|
||||||
row_colors=row_colors,
|
column_widths_cm=col_widths_cm,
|
||||||
column_widths_cm=col_widths_cm,
|
font_size_pt=float(font_pt),
|
||||||
font_size_pt=float(font_pt),
|
autofit=False,
|
||||||
autofit=False,
|
)
|
||||||
)
|
doc.add_page_break()
|
||||||
doc.add_page_break()
|
|
||||||
|
|
||||||
# Appendix
|
# Appendix
|
||||||
_add_title(doc, "Appendix", size_pt=16)
|
_add_title(doc, "Appendix", size_pt=16)
|
||||||
@ -1012,59 +1016,16 @@ def create_docx_report(
|
|||||||
_add_bullets(
|
_add_bullets(
|
||||||
doc,
|
doc,
|
||||||
[
|
[
|
||||||
"Turbines are grouped by proximity (within 4km of each other)",
|
"Farm centroid is provided by the monitoring workflow and shared by every map and table in this report",
|
||||||
"Group centroid = Average latitude and longitude of all turbines in the group",
|
"Distance rings are drawn from the farm centroid; innermost rings represent the highest proximity risk",
|
||||||
"Distance rings are drawn from the group centroid, not individual turbines",
|
"Lightning proximity for each strike is measured from the farm centroid using the Haversine formula",
|
||||||
"Single isolated turbines get their own centroid (individual analysis)",
|
"The monitoring boundary defines the outer analysis radius — strikes beyond it are excluded",
|
||||||
"Distance calculations use Haversine formula for accurate geographic distances",
|
|
||||||
],
|
],
|
||||||
size_pt=10,
|
size_pt=10,
|
||||||
)
|
)
|
||||||
|
|
||||||
# 5. Turbine Grouping Method
|
# 5. Frequent Lightning Activity Period Detection Algorithm
|
||||||
_add_title(doc, "5. Turbine Grouping Method", size_pt=14)
|
_add_title(doc, "5. Frequent Lightning Activity Period Detection Algorithm", size_pt=14)
|
||||||
_add_paragraph(doc, "How Turbines Are Grouped for Analysis:", size_pt=11, bold=True)
|
|
||||||
_add_paragraph(doc, "The system uses an intelligent clustering algorithm to group turbines based on geographic proximity.", size_pt=10)
|
|
||||||
_add_paragraph(doc, "Grouping Algorithm Process:", size_pt=11, bold=True)
|
|
||||||
from src.utils import get_grouping_radius_m
|
|
||||||
grouping_radius_m = get_grouping_radius_m()
|
|
||||||
grouping_radius_km = grouping_radius_m / 1000.0
|
|
||||||
_add_bullets(
|
|
||||||
doc,
|
|
||||||
[
|
|
||||||
"Proximity Analysis: All turbines are analyzed for pairwise distances using Haversine formula",
|
|
||||||
"Clustering Method: Uses DBSCAN (Density-Based Spatial Clustering) algorithm when available",
|
|
||||||
f"Dynamic Distance Threshold: Configurable grouping radius: {grouping_radius_km:.1f}km ({grouping_radius_m}m)",
|
|
||||||
"Fallback Algorithm: If DBSCAN unavailable, uses iterative proximity grouping",
|
|
||||||
"Group Formation: Each group contains turbines that are mutually within the distance threshold",
|
|
||||||
],
|
|
||||||
size_pt=10,
|
|
||||||
)
|
|
||||||
_add_paragraph(doc, "Group Types and Analysis:", size_pt=11, bold=True)
|
|
||||||
_add_bullets(
|
|
||||||
doc,
|
|
||||||
[
|
|
||||||
"Multi-Turbine Groups: Multiple turbines within proximity range - analyzed as a cluster",
|
|
||||||
"Single Turbine Groups: Isolated turbines beyond grouping threshold - individual analysis",
|
|
||||||
"Centroid Calculation: Each group's center point calculated as mean of member coordinates",
|
|
||||||
"Distance Ring Origin: All proximity calculations use group centroid as reference point",
|
|
||||||
],
|
|
||||||
size_pt=10,
|
|
||||||
)
|
|
||||||
_add_paragraph(doc, "Benefits of Grouping Approach:", size_pt=11, bold=True)
|
|
||||||
_add_bullets(
|
|
||||||
doc,
|
|
||||||
[
|
|
||||||
"Efficient Analysis: Reduces computational complexity for large wind farms",
|
|
||||||
"Logical Clustering: Groups turbines that experience similar weather conditions",
|
|
||||||
"Risk Assessment: Provides both group-level and individual turbine risk analysis",
|
|
||||||
"Scalability: Algorithm adapts to wind farms of any size and turbine distribution",
|
|
||||||
],
|
|
||||||
size_pt=10,
|
|
||||||
)
|
|
||||||
|
|
||||||
# 6. Frequent Lightning Activity Period Detection Algorithm
|
|
||||||
_add_title(doc, "6. Frequent Lightning Activity Period Detection Algorithm", size_pt=14)
|
|
||||||
_add_paragraph(doc, "How Period Timespans Are Determined:", size_pt=11, bold=True)
|
_add_paragraph(doc, "How Period Timespans Are Determined:", size_pt=11, bold=True)
|
||||||
_add_paragraph(doc, "The algorithm uses a gap-based approach to identify concentrated lightning activity periods.", size_pt=10)
|
_add_paragraph(doc, "The algorithm uses a gap-based approach to identify concentrated lightning activity periods.", size_pt=10)
|
||||||
_add_paragraph(doc, "Step-by-Step Process:", size_pt=11, bold=True)
|
_add_paragraph(doc, "Step-by-Step Process:", size_pt=11, bold=True)
|
||||||
@ -1081,8 +1042,8 @@ def create_docx_report(
|
|||||||
size_pt=10,
|
size_pt=10,
|
||||||
)
|
)
|
||||||
|
|
||||||
# 7. EarthNetworks Total Lightning Network (ENTLN)
|
# 6. EarthNetworks Total Lightning Network (ENTLN)
|
||||||
_add_title(doc, "7. EarthNetworks Total Lightning Network (ENTLN)", size_pt=14)
|
_add_title(doc, "6. EarthNetworks Total Lightning Network (ENTLN)", size_pt=14)
|
||||||
_entln_blocks = [
|
_entln_blocks = [
|
||||||
"The lightning data presented in this report is sourced directly from the EarthNetworks Total Lightning Network (ENTLN), which monitors both in-cloud and cloud-to-ground strikes. ENTLN is the first in-cloud lightning and cloud-to-ground detection network deployed on a global basis. It includes more than 1,500 sensors (the world's largest) that incorporate advanced systems and technology to provide unmatched in-cloud and cloud-to-ground lightning detection – total lightning detection.",
|
"The lightning data presented in this report is sourced directly from the EarthNetworks Total Lightning Network (ENTLN), which monitors both in-cloud and cloud-to-ground strikes. ENTLN is the first in-cloud lightning and cloud-to-ground detection network deployed on a global basis. It includes more than 1,500 sensors (the world's largest) that incorporate advanced systems and technology to provide unmatched in-cloud and cloud-to-ground lightning detection – total lightning detection.",
|
||||||
"A variety of government agencies, such as the National Weather Service, the Air Force Weather Agency, and organizations that include public safety and emergency response agencies, airports, and utilities rely on information from the ENTLN to make informed decisions.",
|
"A variety of government agencies, such as the National Weather Service, the Air Force Weather Agency, and organizations that include public safety and emergency response agencies, airports, and utilities rely on information from the ENTLN to make informed decisions.",
|
||||||
|
|||||||
@ -1,13 +1,11 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
from typing import Any
|
|
||||||
|
|
||||||
import numpy as np
|
import numpy as np
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
import plotly.graph_objects as go
|
import plotly.graph_objects as go
|
||||||
|
|
||||||
from src.config import config
|
from src.config import config
|
||||||
from src.reporting.precompute import precompute_group_distances_and_rings
|
from src.reporting.precompute import precompute_distances_and_rings
|
||||||
|
|
||||||
|
|
||||||
def _build_current_vs_distance_chart(
|
def _build_current_vs_distance_chart(
|
||||||
@ -125,10 +123,10 @@ def _build_current_vs_distance_chart(
|
|||||||
return fig
|
return fig
|
||||||
|
|
||||||
|
|
||||||
def build_group_lightning_table_data(
|
def build_lightning_event_table_data(
|
||||||
centroid_lat: float, centroid_lng: float, lightning_df: pd.DataFrame
|
centroid_lat: float, centroid_lng: float, lightning_df: pd.DataFrame
|
||||||
) -> tuple[list[list[str]], list[str]]:
|
) -> tuple[list[list[str]], list[str]]:
|
||||||
pre = precompute_group_distances_and_rings(
|
pre = precompute_distances_and_rings(
|
||||||
centroid_lat, centroid_lng, lightning_df, config.distance_rings
|
centroid_lat, centroid_lng, lightning_df, config.distance_rings
|
||||||
)
|
)
|
||||||
rows: list[list[str]] = []
|
rows: list[list[str]] = []
|
||||||
@ -202,18 +200,17 @@ def build_group_lightning_table_data(
|
|||||||
|
|
||||||
|
|
||||||
def build_risk_table_data(
|
def build_risk_table_data(
|
||||||
turbine_df: pd.DataFrame, group_info: dict[str, Any]
|
turbine_df: pd.DataFrame,
|
||||||
) -> tuple[list[list[str]] | None, list[str] | None]:
|
) -> tuple[list[list[str]] | None, list[str] | None]:
|
||||||
if "risk_log" not in turbine_df.columns:
|
if "risk_log" not in turbine_df.columns:
|
||||||
return None, None
|
return None, None
|
||||||
|
|
||||||
group_turbines = turbine_df.iloc[group_info["turbine_indices"]]
|
|
||||||
rows: list[list[str]] = []
|
rows: list[list[str]] = []
|
||||||
row_colors: list[str] = []
|
row_colors: list[str] = []
|
||||||
|
|
||||||
from src.utils import get_risk_definition_by_fixed_intervals, get_turbine_color_by_fixed_intervals
|
from src.utils import get_risk_definition_by_fixed_intervals, get_turbine_color_by_fixed_intervals
|
||||||
|
|
||||||
for _, turbine in group_turbines.iterrows():
|
for _, turbine in turbine_df.iterrows():
|
||||||
risk_log = float(turbine.get("risk_log", 0) or 0)
|
risk_log = float(turbine.get("risk_log", 0) or 0)
|
||||||
color = get_turbine_color_by_fixed_intervals(risk_log)
|
color = get_turbine_color_by_fixed_intervals(risk_log)
|
||||||
rows.append(
|
rows.append(
|
||||||
|
|||||||
@ -1,63 +1,10 @@
|
|||||||
import numpy as np
|
import numpy as np
|
||||||
import pandas as pd
|
import pandas as pd
|
||||||
from typing import List, Tuple, Optional, Dict, Any
|
from typing import List, Dict, Any
|
||||||
from src.analysis.geospatial import haversine_distance_vectorized
|
from src.analysis.geospatial import haversine_distance_vectorized
|
||||||
from src.utils import format_datetime_ddmmyyyy_hhmmss
|
|
||||||
|
|
||||||
def build_lightning_table_rows(
|
|
||||||
centroid_lat: float,
|
|
||||||
centroid_lng: float,
|
|
||||||
lightning_df: pd.DataFrame,
|
|
||||||
distance_rings_m: List[int],
|
|
||||||
ring_colors: List[str],
|
|
||||||
) -> Tuple[List[List[str]], List[str]]:
|
|
||||||
rows: List[List[str]] = []
|
|
||||||
row_colors: List[str] = []
|
|
||||||
|
|
||||||
if len(lightning_df) == 0:
|
|
||||||
return rows, row_colors
|
|
||||||
|
|
||||||
outermost_km = max(distance_rings_m) / 1000.0
|
|
||||||
rings_km = np.array(distance_rings_m, dtype=float) / 1000.0
|
|
||||||
|
|
||||||
dists_km = haversine_distance_vectorized(
|
|
||||||
centroid_lat, centroid_lng,
|
|
||||||
lightning_df['lat'].values, lightning_df['lng'].values,
|
|
||||||
) / 1000.0
|
|
||||||
|
|
||||||
for i, rec in enumerate(lightning_df.itertuples(index=False)):
|
|
||||||
proximity = dists_km[i]
|
|
||||||
if proximity > outermost_km:
|
|
||||||
continue
|
|
||||||
|
|
||||||
ring_idx = int(np.searchsorted(rings_km, proximity, side='left'))
|
|
||||||
if ring_idx >= len(rings_km):
|
|
||||||
continue
|
|
||||||
color = ring_colors[ring_idx]
|
|
||||||
|
|
||||||
try:
|
|
||||||
dt_val = rec.local_time if not isinstance(rec.local_time, str) else pd.to_datetime(str(rec.local_time)[:19])
|
|
||||||
local_time = format_datetime_ddmmyyyy_hhmmss(pd.to_datetime(dt_val))
|
|
||||||
except Exception:
|
|
||||||
local_time = str(getattr(rec, 'local_time', ''))[:19]
|
|
||||||
|
|
||||||
lightning_type = "cloud-to-ground" if str(rec.p_type) == '0' else "intercloud"
|
|
||||||
rows.append([
|
|
||||||
"",
|
|
||||||
local_time,
|
|
||||||
f"{rec.lat:.5f}",
|
|
||||||
f"{rec.lng:.5f}",
|
|
||||||
str(rec.current),
|
|
||||||
str(getattr(rec, 'height', '')),
|
|
||||||
lightning_type,
|
|
||||||
f"{proximity:.2f}"
|
|
||||||
])
|
|
||||||
row_colors.append(color)
|
|
||||||
|
|
||||||
return rows, row_colors
|
|
||||||
|
|
||||||
|
|
||||||
def precompute_group_distances_and_rings(
|
def precompute_distances_and_rings(
|
||||||
centroid_lat: float,
|
centroid_lat: float,
|
||||||
centroid_lng: float,
|
centroid_lng: float,
|
||||||
lightning_df: pd.DataFrame,
|
lightning_df: pd.DataFrame,
|
||||||
|
|||||||
21
src/utils.py
21
src/utils.py
@ -128,26 +128,13 @@ def format_period_display_for_report(start_value: Optional[str], end_value: Opti
|
|||||||
end_display = _format_one(end_value) if end_value else ""
|
end_display = _format_one(end_value) if end_value else ""
|
||||||
return start_display, end_display
|
return start_display, end_display
|
||||||
|
|
||||||
def get_grouping_radius_m() -> int:
|
|
||||||
from .config import config
|
|
||||||
rings = config.distance_rings or []
|
|
||||||
grouping = config.grouping_params or {}
|
|
||||||
max_distance_m = grouping.get('max_distance_m')
|
|
||||||
if isinstance(max_distance_m, (int, float)) and max_distance_m > 0:
|
|
||||||
return int(max_distance_m)
|
|
||||||
ring_index = grouping.get('distance_ring_index', 2)
|
|
||||||
if isinstance(ring_index, int) and 0 <= ring_index < len(rings):
|
|
||||||
return int(rings[ring_index])
|
|
||||||
return int(rings[2] if len(rings) > 2 else (max(rings) if rings else 0))
|
|
||||||
|
|
||||||
def get_analysis_radius_m() -> int:
|
def get_analysis_radius_m() -> int:
|
||||||
from .config import config
|
from .config import config
|
||||||
|
boundary = config.analysis_boundary_m
|
||||||
|
if isinstance(boundary, (int, float)) and boundary > 0:
|
||||||
|
return int(boundary)
|
||||||
rings = config.distance_rings or []
|
rings = config.distance_rings or []
|
||||||
grouping = config.grouping_params or {}
|
return int(max(rings)) if rings else 0
|
||||||
ring_index = grouping.get('distance_ring_index')
|
|
||||||
if isinstance(ring_index, int) and 0 <= ring_index < len(rings):
|
|
||||||
return int(rings[ring_index])
|
|
||||||
return int(max(rings) if rings else 0)
|
|
||||||
|
|
||||||
def get_turbine_color_by_fixed_intervals(risk_log_value: float) -> str:
|
def get_turbine_color_by_fixed_intervals(risk_log_value: float) -> str:
|
||||||
"""
|
"""
|
||||||
|
|||||||
@ -1,77 +0,0 @@
|
|||||||
[
|
|
||||||
{
|
|
||||||
"name": "T1",
|
|
||||||
"lat": 36.771944,
|
|
||||||
"lng": 33.405000
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T2",
|
|
||||||
"lat": 36.770278,
|
|
||||||
"lng": 33.407500
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T3",
|
|
||||||
"lat": 36.768611,
|
|
||||||
"lng": 33.410000
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T4",
|
|
||||||
"lat": 36.766389,
|
|
||||||
"lng": 33.412500
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T5",
|
|
||||||
"lat": 36.764167,
|
|
||||||
"lng": 33.414722
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T6",
|
|
||||||
"lat": 36.762778,
|
|
||||||
"lng": 33.417500
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T7",
|
|
||||||
"lat": 36.772778,
|
|
||||||
"lng": 33.411667
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T8",
|
|
||||||
"lat": 36.770556,
|
|
||||||
"lng": 33.414167
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T9",
|
|
||||||
"lat": 36.761667,
|
|
||||||
"lng": 33.420000
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T10",
|
|
||||||
"lat": 36.766944,
|
|
||||||
"lng": 33.419167
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T11",
|
|
||||||
"lat": 36.765833,
|
|
||||||
"lng": 33.421667
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T12",
|
|
||||||
"lat": 36.773889,
|
|
||||||
"lng": 33.416389
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T13",
|
|
||||||
"lat": 36.774722,
|
|
||||||
"lng": 33.420556
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T14",
|
|
||||||
"lat": 36.773333,
|
|
||||||
"lng": 33.401111
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "T15",
|
|
||||||
"lat": 36.759722,
|
|
||||||
"lng": 33.422778
|
|
||||||
}
|
|
||||||
]
|
|
||||||
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
File diff suppressed because one or more lines are too long
File diff suppressed because it is too large
Load Diff
@ -1,2 +0,0 @@
|
|||||||
pyproj>=3.0.0
|
|
||||||
pandas>=1.3.0
|
|
||||||
@ -1,262 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""
|
|
||||||
UTM ED50 to WGS84 Coordinate Converter
|
|
||||||
|
|
||||||
This script converts UTM coordinates in 6-degree zones from ED50 (European Datum 1950)
|
|
||||||
reference system to WGS84 format.
|
|
||||||
|
|
||||||
Requirements:
|
|
||||||
pip install pyproj pandas
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
python utm_ed50_to_wgs84_converter.py input_file.csv output_file.csv
|
|
||||||
python utm_ed50_to_wgs84_converter.py --interactive
|
|
||||||
"""
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import pandas as pd
|
|
||||||
import pyproj
|
|
||||||
from typing import Tuple, List, Optional
|
|
||||||
import sys
|
|
||||||
|
|
||||||
|
|
||||||
class UTMED50ToWGS84Converter:
|
|
||||||
"""Converter for UTM ED50 coordinates to WGS84."""
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
"""Initialize the converter with ED50 and WGS84 projections."""
|
|
||||||
self.ed50_utm_projections = {}
|
|
||||||
self.wgs84_utm_projections = {}
|
|
||||||
|
|
||||||
def get_ed50_utm_projection(self, zone: int, northern: bool = True) -> pyproj.Proj:
|
|
||||||
"""Get ED50 UTM projection for a specific zone."""
|
|
||||||
key = (zone, northern)
|
|
||||||
if key not in self.ed50_utm_projections:
|
|
||||||
hemisphere = 'N' if northern else 'S'
|
|
||||||
proj_string = f"+proj=utm +zone={zone} +ellps=intl +towgs84=-87,-98,-121,0,0,0,0 +units=m +no_defs"
|
|
||||||
self.ed50_utm_projections[key] = pyproj.Proj(proj_string)
|
|
||||||
return self.ed50_utm_projections[key]
|
|
||||||
|
|
||||||
def get_wgs84_utm_projection(self, zone: int, northern: bool = True) -> pyproj.Proj:
|
|
||||||
"""Get WGS84 UTM projection for a specific zone."""
|
|
||||||
key = (zone, northern)
|
|
||||||
if key not in self.wgs84_utm_projections:
|
|
||||||
hemisphere = 'N' if northern else 'S'
|
|
||||||
proj_string = f"+proj=utm +zone={zone} +ellps=WGS84 +datum=WGS84 +units=m +no_defs"
|
|
||||||
self.wgs84_utm_projections[key] = pyproj.Proj(proj_string)
|
|
||||||
return self.wgs84_utm_projections[key]
|
|
||||||
|
|
||||||
def convert_single_point(self, easting: float, northing: float, zone: int,
|
|
||||||
northern: bool = True) -> Tuple[float, float]:
|
|
||||||
"""
|
|
||||||
Convert a single UTM ED50 point to WGS84 lat/lon.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
easting: UTM easting coordinate in meters
|
|
||||||
northing: UTM northing coordinate in meters
|
|
||||||
zone: UTM zone number (1-60)
|
|
||||||
northern: True if in northern hemisphere, False if southern
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Tuple of (latitude, longitude) in WGS84 decimal degrees
|
|
||||||
"""
|
|
||||||
if not (1 <= zone <= 60):
|
|
||||||
raise ValueError(f"Invalid UTM zone: {zone}. Must be between 1 and 60.")
|
|
||||||
|
|
||||||
# Create ED50 UTM projection
|
|
||||||
ed50_proj = self.get_ed50_utm_projection(zone, northern)
|
|
||||||
|
|
||||||
# Create WGS84 lat/lon projection
|
|
||||||
wgs84_latlon = pyproj.Proj('+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs')
|
|
||||||
|
|
||||||
# Create transformer from ED50 UTM to WGS84 lat/lon
|
|
||||||
transformer = pyproj.Transformer.from_proj(ed50_proj, wgs84_latlon)
|
|
||||||
|
|
||||||
# Transform coordinates directly to WGS84 lat/lon
|
|
||||||
wgs84_lon, wgs84_lat = transformer.transform(easting, northing)
|
|
||||||
|
|
||||||
return wgs84_lat, wgs84_lon
|
|
||||||
|
|
||||||
def convert_dataframe(self, df: pd.DataFrame, easting_col: str, northing_col: str,
|
|
||||||
zone_col: str, northern_col: Optional[str] = None,
|
|
||||||
northern_default: bool = True) -> pd.DataFrame:
|
|
||||||
"""
|
|
||||||
Convert UTM ED50 coordinates in a DataFrame to WGS84.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
df: Input DataFrame with UTM coordinates
|
|
||||||
easting_col: Column name for easting coordinates
|
|
||||||
northing_col: Column name for northing coordinates
|
|
||||||
zone_col: Column name for UTM zone
|
|
||||||
northern_col: Column name for northern hemisphere flag (optional)
|
|
||||||
northern_default: Default value for northern hemisphere if column not provided
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
DataFrame with additional WGS84 lat/lon columns
|
|
||||||
"""
|
|
||||||
result_df = df.copy()
|
|
||||||
result_df['wgs84_lat'] = None
|
|
||||||
result_df['wgs84_lon'] = None
|
|
||||||
|
|
||||||
for idx, row in df.iterrows():
|
|
||||||
try:
|
|
||||||
easting = float(row[easting_col])
|
|
||||||
northing = float(row[northing_col])
|
|
||||||
zone = int(row[zone_col])
|
|
||||||
|
|
||||||
if northern_col and northern_col in row:
|
|
||||||
northern = bool(row[northern_col])
|
|
||||||
else:
|
|
||||||
northern = northern_default
|
|
||||||
|
|
||||||
lat, lon = self.convert_single_point(easting, northing, zone, northern)
|
|
||||||
result_df.at[idx, 'wgs84_lat'] = lat
|
|
||||||
result_df.at[idx, 'wgs84_lon'] = lon
|
|
||||||
|
|
||||||
except (ValueError, KeyError) as e:
|
|
||||||
print(f"Warning: Could not convert row {idx}: {e}")
|
|
||||||
result_df.at[idx, 'wgs84_lat'] = None
|
|
||||||
result_df.at[idx, 'wgs84_lon'] = None
|
|
||||||
|
|
||||||
return result_df
|
|
||||||
|
|
||||||
|
|
||||||
def interactive_mode():
|
|
||||||
"""Run the converter in interactive mode."""
|
|
||||||
converter = UTMED50ToWGS84Converter()
|
|
||||||
|
|
||||||
print("UTM ED50 to WGS84 Coordinate Converter")
|
|
||||||
print("=" * 40)
|
|
||||||
print("Enter coordinates (type 'quit' to exit)")
|
|
||||||
print()
|
|
||||||
|
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
# Get input
|
|
||||||
easting_input = input("Enter easting (meters): ").strip()
|
|
||||||
if easting_input.lower() == 'quit':
|
|
||||||
break
|
|
||||||
|
|
||||||
northing_input = input("Enter northing (meters): ").strip()
|
|
||||||
if northing_input.lower() == 'quit':
|
|
||||||
break
|
|
||||||
|
|
||||||
zone_input = input("Enter UTM zone (1-60): ").strip()
|
|
||||||
if zone_input.lower() == 'quit':
|
|
||||||
break
|
|
||||||
|
|
||||||
hemisphere_input = input("Enter hemisphere (N/S) [default: N]: ").strip().upper()
|
|
||||||
if hemisphere_input == 'QUIT':
|
|
||||||
break
|
|
||||||
|
|
||||||
# Parse inputs
|
|
||||||
easting = float(easting_input)
|
|
||||||
northing = float(northing_input)
|
|
||||||
zone = int(zone_input)
|
|
||||||
northern = hemisphere_input != 'S' if hemisphere_input else True
|
|
||||||
|
|
||||||
# Convert
|
|
||||||
lat, lon = converter.convert_single_point(easting, northing, zone, northern)
|
|
||||||
|
|
||||||
print(f"\nWGS84 Coordinates:")
|
|
||||||
print(f"Latitude: {lat:.8f}°")
|
|
||||||
print(f"Longitude: {lon:.8f}°")
|
|
||||||
print("-" * 40)
|
|
||||||
|
|
||||||
except ValueError as e:
|
|
||||||
print(f"Error: {e}")
|
|
||||||
print("Please enter valid numeric values.")
|
|
||||||
except KeyboardInterrupt:
|
|
||||||
print("\nExiting...")
|
|
||||||
break
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
"""Main function to handle command line arguments and file processing."""
|
|
||||||
parser = argparse.ArgumentParser(
|
|
||||||
description="Convert UTM ED50 coordinates to WGS84 format",
|
|
||||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
|
||||||
epilog="""
|
|
||||||
Examples:
|
|
||||||
# Interactive mode
|
|
||||||
python utm_ed50_to_wgs84_converter.py --interactive
|
|
||||||
|
|
||||||
# Convert CSV file
|
|
||||||
python utm_ed50_to_wgs84_converter.py input.csv output.csv
|
|
||||||
|
|
||||||
# Convert with custom column names
|
|
||||||
python utm_ed50_to_wgs84_converter.py input.csv output.csv --easting-col X --northing-col Y --zone-col ZONE
|
|
||||||
"""
|
|
||||||
)
|
|
||||||
|
|
||||||
parser.add_argument('input_file', nargs='?', help='Input CSV file with UTM coordinates')
|
|
||||||
parser.add_argument('output_file', nargs='?', help='Output CSV file for WGS84 coordinates')
|
|
||||||
parser.add_argument('--interactive', '-i', action='store_true',
|
|
||||||
help='Run in interactive mode')
|
|
||||||
parser.add_argument('--easting-col', default='easting',
|
|
||||||
help='Column name for easting coordinates (default: easting)')
|
|
||||||
parser.add_argument('--northing-col', default='northing',
|
|
||||||
help='Column name for northing coordinates (default: northing)')
|
|
||||||
parser.add_argument('--zone-col', default='zone',
|
|
||||||
help='Column name for UTM zone (default: zone)')
|
|
||||||
parser.add_argument('--northern-col',
|
|
||||||
help='Column name for northern hemisphere flag (optional)')
|
|
||||||
parser.add_argument('--northern-default', action='store_true', default=True,
|
|
||||||
help='Default value for northern hemisphere (default: True)')
|
|
||||||
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
if args.interactive:
|
|
||||||
interactive_mode()
|
|
||||||
return
|
|
||||||
|
|
||||||
if not args.input_file or not args.output_file:
|
|
||||||
parser.error("Both input_file and output_file are required when not in interactive mode")
|
|
||||||
|
|
||||||
try:
|
|
||||||
# Read input file
|
|
||||||
print(f"Reading input file: {args.input_file}")
|
|
||||||
df = pd.read_csv(args.input_file)
|
|
||||||
|
|
||||||
# Check required columns
|
|
||||||
required_cols = [args.easting_col, args.northing_col, args.zone_col]
|
|
||||||
missing_cols = [col for col in required_cols if col not in df.columns]
|
|
||||||
if missing_cols:
|
|
||||||
print(f"Error: Missing required columns: {missing_cols}")
|
|
||||||
print(f"Available columns: {list(df.columns)}")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
# Convert coordinates
|
|
||||||
print("Converting coordinates...")
|
|
||||||
converter = UTMED50ToWGS84Converter()
|
|
||||||
result_df = converter.convert_dataframe(
|
|
||||||
df,
|
|
||||||
args.easting_col,
|
|
||||||
args.northing_col,
|
|
||||||
args.zone_col,
|
|
||||||
args.northern_col,
|
|
||||||
args.northern_default
|
|
||||||
)
|
|
||||||
|
|
||||||
# Save output
|
|
||||||
print(f"Saving output file: {args.output_file}")
|
|
||||||
result_df.to_csv(args.output_file, index=False)
|
|
||||||
|
|
||||||
# Print summary
|
|
||||||
total_rows = len(result_df)
|
|
||||||
successful_conversions = result_df['wgs84_lat'].notna().sum()
|
|
||||||
print(f"\nConversion complete!")
|
|
||||||
print(f"Total rows: {total_rows}")
|
|
||||||
print(f"Successful conversions: {successful_conversions}")
|
|
||||||
print(f"Failed conversions: {total_rows - successful_conversions}")
|
|
||||||
|
|
||||||
except FileNotFoundError:
|
|
||||||
print(f"Error: Input file '{args.input_file}' not found")
|
|
||||||
sys.exit(1)
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Error: {e}")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@ -1,340 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
"""
|
|
||||||
Enhanced UTM ED50 to WGS84 Coordinate Converter
|
|
||||||
|
|
||||||
This script converts UTM coordinates in 6-degree zones from ED50 (European Datum 1950)
|
|
||||||
reference system to WGS84 format. Supports both CSV and JSON input files.
|
|
||||||
|
|
||||||
Requirements:
|
|
||||||
pip install pyproj pandas
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
python utm_ed50_to_wgs84_converter_enhanced.py input_file.csv output_file.csv
|
|
||||||
python utm_ed50_to_wgs84_converter_enhanced.py input_file.json output_file.json
|
|
||||||
python utm_ed50_to_wgs84_converter_enhanced.py --interactive
|
|
||||||
"""
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import pandas as pd
|
|
||||||
import pyproj
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
from typing import Tuple, List, Optional, Union
|
|
||||||
import sys
|
|
||||||
|
|
||||||
|
|
||||||
class UTMED50ToWGS84Converter:
|
|
||||||
"""Converter for UTM ED50 coordinates to WGS84."""
|
|
||||||
|
|
||||||
def __init__(self):
|
|
||||||
"""Initialize the converter with ED50 and WGS84 projections."""
|
|
||||||
self.ed50_utm_projections = {}
|
|
||||||
self.wgs84_utm_projections = {}
|
|
||||||
|
|
||||||
def get_ed50_utm_projection(self, zone: int, northern: bool = True) -> pyproj.Proj:
|
|
||||||
"""Get ED50 UTM projection for a specific zone."""
|
|
||||||
key = (zone, northern)
|
|
||||||
if key not in self.ed50_utm_projections:
|
|
||||||
hemisphere = 'N' if northern else 'S'
|
|
||||||
proj_string = f"+proj=utm +zone={zone} +ellps=intl +towgs84=-87,-98,-121,0,0,0,0 +units=m +no_defs"
|
|
||||||
self.ed50_utm_projections[key] = pyproj.Proj(proj_string)
|
|
||||||
return self.ed50_utm_projections[key]
|
|
||||||
|
|
||||||
def get_wgs84_utm_projection(self, zone: int, northern: bool = True) -> pyproj.Proj:
|
|
||||||
"""Get WGS84 UTM projection for a specific zone."""
|
|
||||||
key = (zone, northern)
|
|
||||||
if key not in self.wgs84_utm_projections:
|
|
||||||
hemisphere = 'N' if northern else 'S'
|
|
||||||
proj_string = f"+proj=utm +zone={zone} +ellps=WGS84 +datum=WGS84 +units=m +no_defs"
|
|
||||||
self.wgs84_utm_projections[key] = pyproj.Proj(proj_string)
|
|
||||||
return self.wgs84_utm_projections[key]
|
|
||||||
|
|
||||||
def convert_single_point(self, easting: float, northing: float, zone: int,
|
|
||||||
northern: bool = True) -> Tuple[float, float]:
|
|
||||||
"""
|
|
||||||
Convert a single UTM ED50 point to WGS84 lat/lon.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
easting: UTM easting coordinate in meters
|
|
||||||
northing: UTM northing coordinate in meters
|
|
||||||
zone: UTM zone number (1-60)
|
|
||||||
northern: True if in northern hemisphere, False if southern
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
Tuple of (latitude, longitude) in WGS84 decimal degrees
|
|
||||||
"""
|
|
||||||
if not (1 <= zone <= 60):
|
|
||||||
raise ValueError(f"Invalid UTM zone: {zone}. Must be between 1 and 60.")
|
|
||||||
|
|
||||||
# Create ED50 UTM projection
|
|
||||||
ed50_proj = self.get_ed50_utm_projection(zone, northern)
|
|
||||||
|
|
||||||
# Create WGS84 lat/lon projection
|
|
||||||
wgs84_latlon = pyproj.Proj('+proj=longlat +ellps=WGS84 +datum=WGS84 +no_defs')
|
|
||||||
|
|
||||||
# Create transformer from ED50 UTM to WGS84 lat/lon
|
|
||||||
transformer = pyproj.Transformer.from_proj(ed50_proj, wgs84_latlon)
|
|
||||||
|
|
||||||
# Transform coordinates directly to WGS84 lat/lon
|
|
||||||
wgs84_lon, wgs84_lat = transformer.transform(easting, northing)
|
|
||||||
|
|
||||||
return wgs84_lat, wgs84_lon
|
|
||||||
|
|
||||||
def convert_dataframe(self, df: pd.DataFrame, easting_col: str, northing_col: str,
|
|
||||||
zone_col: str, northern_col: Optional[str] = None,
|
|
||||||
northern_default: bool = True) -> pd.DataFrame:
|
|
||||||
"""
|
|
||||||
Convert UTM ED50 coordinates in a DataFrame to WGS84.
|
|
||||||
|
|
||||||
Args:
|
|
||||||
df: Input DataFrame with UTM coordinates
|
|
||||||
easting_col: Column name for easting coordinates
|
|
||||||
northing_col: Column name for northing coordinates
|
|
||||||
zone_col: Column name for UTM zone
|
|
||||||
northern_col: Column name for northern hemisphere flag (optional)
|
|
||||||
northern_default: Default value for northern hemisphere if column not provided
|
|
||||||
|
|
||||||
Returns:
|
|
||||||
DataFrame with additional lat/lng columns and renamed description column
|
|
||||||
"""
|
|
||||||
result_df = df.copy()
|
|
||||||
result_df['lat'] = None
|
|
||||||
result_df['lng'] = None
|
|
||||||
|
|
||||||
# Rename description column to name if it exists
|
|
||||||
if 'description' in result_df.columns:
|
|
||||||
result_df = result_df.rename(columns={'description': 'name'})
|
|
||||||
|
|
||||||
for idx, row in df.iterrows():
|
|
||||||
try:
|
|
||||||
easting = float(row[easting_col])
|
|
||||||
northing = float(row[northing_col])
|
|
||||||
zone = int(row[zone_col])
|
|
||||||
|
|
||||||
if northern_col and northern_col in row:
|
|
||||||
northern = bool(row[northern_col])
|
|
||||||
else:
|
|
||||||
northern = northern_default
|
|
||||||
|
|
||||||
lat, lon = self.convert_single_point(easting, northing, zone, northern)
|
|
||||||
result_df.at[idx, 'lat'] = lat
|
|
||||||
result_df.at[idx, 'lng'] = lon
|
|
||||||
|
|
||||||
except (ValueError, KeyError) as e:
|
|
||||||
print(f"Warning: Could not convert row {idx}: {e}")
|
|
||||||
result_df.at[idx, 'lat'] = None
|
|
||||||
result_df.at[idx, 'lng'] = None
|
|
||||||
|
|
||||||
return result_df
|
|
||||||
|
|
||||||
|
|
||||||
def detect_file_format(file_path: str) -> str:
|
|
||||||
"""Detect if file is CSV or JSON based on extension and content."""
|
|
||||||
_, ext = os.path.splitext(file_path.lower())
|
|
||||||
|
|
||||||
if ext == '.csv':
|
|
||||||
return 'csv'
|
|
||||||
elif ext == '.json':
|
|
||||||
return 'json'
|
|
||||||
else:
|
|
||||||
# Try to detect by reading first few lines
|
|
||||||
try:
|
|
||||||
with open(file_path, 'r', encoding='utf-8') as f:
|
|
||||||
first_line = f.readline().strip()
|
|
||||||
if first_line.startswith('[') or first_line.startswith('{'):
|
|
||||||
return 'json'
|
|
||||||
else:
|
|
||||||
return 'csv'
|
|
||||||
except:
|
|
||||||
return 'csv' # Default to CSV
|
|
||||||
|
|
||||||
|
|
||||||
def load_data(file_path: str, csv_separator: str = ';', decimal_separator: str = ',') -> pd.DataFrame:
|
|
||||||
"""Load data from CSV or JSON file."""
|
|
||||||
file_format = detect_file_format(file_path)
|
|
||||||
|
|
||||||
if file_format == 'json':
|
|
||||||
with open(file_path, 'r', encoding='utf-8') as f:
|
|
||||||
data = json.load(f)
|
|
||||||
|
|
||||||
if isinstance(data, list):
|
|
||||||
return pd.DataFrame(data)
|
|
||||||
elif isinstance(data, dict):
|
|
||||||
return pd.DataFrame([data])
|
|
||||||
else:
|
|
||||||
raise ValueError("JSON file must contain a list or dictionary")
|
|
||||||
|
|
||||||
else: # CSV
|
|
||||||
return pd.read_csv(file_path, sep=csv_separator, decimal=decimal_separator)
|
|
||||||
|
|
||||||
|
|
||||||
def save_data(df: pd.DataFrame, file_path: str, file_format: str = None, csv_separator: str = ';', decimal_separator: str = ','):
|
|
||||||
"""Save data to CSV or JSON file."""
|
|
||||||
if file_format is None:
|
|
||||||
file_format = detect_file_format(file_path)
|
|
||||||
|
|
||||||
if file_format == 'json':
|
|
||||||
# Convert DataFrame to list of dictionaries
|
|
||||||
data = df.to_dict('records')
|
|
||||||
with open(file_path, 'w', encoding='utf-8') as f:
|
|
||||||
json.dump(data, f, indent=2, ensure_ascii=False)
|
|
||||||
|
|
||||||
else: # CSV
|
|
||||||
df.to_csv(file_path, index=False, sep=csv_separator, decimal=decimal_separator)
|
|
||||||
|
|
||||||
|
|
||||||
def interactive_mode():
|
|
||||||
"""Run the converter in interactive mode."""
|
|
||||||
converter = UTMED50ToWGS84Converter()
|
|
||||||
|
|
||||||
print("UTM ED50 to WGS84 Coordinate Converter")
|
|
||||||
print("=" * 40)
|
|
||||||
print("Enter coordinates (type 'quit' to exit)")
|
|
||||||
print()
|
|
||||||
|
|
||||||
while True:
|
|
||||||
try:
|
|
||||||
# Get input
|
|
||||||
easting_input = input("Enter easting (meters): ").strip()
|
|
||||||
if easting_input.lower() == 'quit':
|
|
||||||
break
|
|
||||||
|
|
||||||
northing_input = input("Enter northing (meters): ").strip()
|
|
||||||
if northing_input.lower() == 'quit':
|
|
||||||
break
|
|
||||||
|
|
||||||
zone_input = input("Enter UTM zone (1-60): ").strip()
|
|
||||||
if zone_input.lower() == 'quit':
|
|
||||||
break
|
|
||||||
|
|
||||||
hemisphere_input = input("Enter hemisphere (N/S) [default: N]: ").strip().upper()
|
|
||||||
if hemisphere_input == 'QUIT':
|
|
||||||
break
|
|
||||||
|
|
||||||
# Parse inputs
|
|
||||||
easting = float(easting_input)
|
|
||||||
northing = float(northing_input)
|
|
||||||
zone = int(zone_input)
|
|
||||||
northern = hemisphere_input != 'S' if hemisphere_input else True
|
|
||||||
|
|
||||||
# Convert
|
|
||||||
lat, lon = converter.convert_single_point(easting, northing, zone, northern)
|
|
||||||
|
|
||||||
print(f"\nWGS84 Coordinates:")
|
|
||||||
print(f"Latitude: {lat:.8f}°")
|
|
||||||
print(f"Longitude: {lon:.8f}°")
|
|
||||||
print("-" * 40)
|
|
||||||
|
|
||||||
except ValueError as e:
|
|
||||||
print(f"Error: {e}")
|
|
||||||
print("Please enter valid numeric values.")
|
|
||||||
except KeyboardInterrupt:
|
|
||||||
print("\nExiting...")
|
|
||||||
break
|
|
||||||
|
|
||||||
|
|
||||||
def main():
|
|
||||||
"""Main function to handle command line arguments and file processing."""
|
|
||||||
parser = argparse.ArgumentParser(
|
|
||||||
description="Convert UTM ED50 coordinates to WGS84 format (supports CSV and JSON)",
|
|
||||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
|
||||||
epilog="""
|
|
||||||
Examples:
|
|
||||||
# Interactive mode
|
|
||||||
python utm_ed50_to_wgs84_converter_enhanced.py --interactive
|
|
||||||
|
|
||||||
# Convert CSV file (semicolon-separated by default)
|
|
||||||
python utm_ed50_to_wgs84_converter_enhanced.py input.csv output.csv
|
|
||||||
|
|
||||||
# Convert CSV file with comma separator
|
|
||||||
python utm_ed50_to_wgs84_converter_enhanced.py input.csv output.csv --separator ,
|
|
||||||
|
|
||||||
# Convert JSON file
|
|
||||||
python utm_ed50_to_wgs84_converter_enhanced.py input.json output.json
|
|
||||||
|
|
||||||
# Convert with custom column names
|
|
||||||
python utm_ed50_to_wgs84_converter_enhanced.py input.csv output.csv --easting-col X --northing-col Y --zone-col ZONE
|
|
||||||
|
|
||||||
# Using full paths
|
|
||||||
python /full/path/to/script.py /full/path/to/input.csv /full/path/to/output.csv
|
|
||||||
"""
|
|
||||||
)
|
|
||||||
|
|
||||||
parser.add_argument('input_file', nargs='?', help='Input file with UTM coordinates (CSV or JSON)')
|
|
||||||
parser.add_argument('output_file', nargs='?', help='Output file for WGS84 coordinates (CSV or JSON)')
|
|
||||||
parser.add_argument('--interactive', '-i', action='store_true',
|
|
||||||
help='Run in interactive mode')
|
|
||||||
parser.add_argument('--easting-col', default='easting',
|
|
||||||
help='Column name for easting coordinates (default: easting)')
|
|
||||||
parser.add_argument('--northing-col', default='northing',
|
|
||||||
help='Column name for northing coordinates (default: northing)')
|
|
||||||
parser.add_argument('--zone-col', default='zone',
|
|
||||||
help='Column name for UTM zone (default: zone)')
|
|
||||||
parser.add_argument('--northern-col',
|
|
||||||
help='Column name for northern hemisphere flag (optional)')
|
|
||||||
parser.add_argument('--northern-default', action='store_true', default=True,
|
|
||||||
help='Default value for northern hemisphere (default: True)')
|
|
||||||
parser.add_argument('--separator', '-s', default=';',
|
|
||||||
help='CSV separator character (default: ;)')
|
|
||||||
parser.add_argument('--decimal', '-d', default=',',
|
|
||||||
help='CSV decimal separator character (default: ,)')
|
|
||||||
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
if args.interactive:
|
|
||||||
interactive_mode()
|
|
||||||
return
|
|
||||||
|
|
||||||
if not args.input_file or not args.output_file:
|
|
||||||
parser.error("Both input_file and output_file are required when not in interactive mode")
|
|
||||||
|
|
||||||
try:
|
|
||||||
# Check if input file exists
|
|
||||||
if not os.path.exists(args.input_file):
|
|
||||||
print(f"Error: Input file '{args.input_file}' not found")
|
|
||||||
print(f"Current working directory: {os.getcwd()}")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
# Load data
|
|
||||||
print(f"Reading input file: {args.input_file}")
|
|
||||||
df = load_data(args.input_file, args.separator, args.decimal)
|
|
||||||
|
|
||||||
# Check required columns
|
|
||||||
required_cols = [args.easting_col, args.northing_col, args.zone_col]
|
|
||||||
missing_cols = [col for col in required_cols if col not in df.columns]
|
|
||||||
if missing_cols:
|
|
||||||
print(f"Error: Missing required columns: {missing_cols}")
|
|
||||||
print(f"Available columns: {list(df.columns)}")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
# Convert coordinates
|
|
||||||
print("Converting coordinates...")
|
|
||||||
converter = UTMED50ToWGS84Converter()
|
|
||||||
result_df = converter.convert_dataframe(
|
|
||||||
df,
|
|
||||||
args.easting_col,
|
|
||||||
args.northing_col,
|
|
||||||
args.zone_col,
|
|
||||||
args.northern_col,
|
|
||||||
args.northern_default
|
|
||||||
)
|
|
||||||
|
|
||||||
# Save output
|
|
||||||
print(f"Saving output file: {args.output_file}")
|
|
||||||
save_data(result_df, args.output_file, csv_separator=args.separator, decimal_separator=args.decimal)
|
|
||||||
|
|
||||||
# Print summary
|
|
||||||
total_rows = len(result_df)
|
|
||||||
successful_conversions = result_df['lat'].notna().sum()
|
|
||||||
print(f"\nConversion complete!")
|
|
||||||
print(f"Total rows: {total_rows}")
|
|
||||||
print(f"Successful conversions: {successful_conversions}")
|
|
||||||
print(f"Failed conversions: {total_rows - successful_conversions}")
|
|
||||||
|
|
||||||
except Exception as e:
|
|
||||||
print(f"Error: {e}")
|
|
||||||
sys.exit(1)
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
main()
|
|
||||||
@ -1,184 +0,0 @@
|
|||||||
{
|
|
||||||
"api_config": {
|
|
||||||
"base_url": "https://risk.tarla.io/api",
|
|
||||||
"timeout_seconds": 60,
|
|
||||||
"retry_attempts": 3,
|
|
||||||
"default_query_range": {
|
|
||||||
"method": "current_month"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"output_base_directory": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/reports/",
|
|
||||||
"default_padding_km": 5,
|
|
||||||
"wind_farms": [
|
|
||||||
{
|
|
||||||
"farm_id": "dagpazari_RES",
|
|
||||||
"name": "Dağpazarı RES",
|
|
||||||
"enabled": true,
|
|
||||||
"coordinates_file": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/coordinates/dagpazari_RES_coordinates.json",
|
|
||||||
"distance_rings": [2000, 4000, 6000, 8000],
|
|
||||||
"ring_colors": ["#B71C1C", "#F94144", "#F8961E", "#90BE6D"],
|
|
||||||
"lightning_source_type": "api",
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "auto",
|
|
||||||
"padding_km": 5
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "manual",
|
|
||||||
"start_date": "01-02-2026",
|
|
||||||
"end_date": "28-02-2026"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/reports/dagpazari_RES/",
|
|
||||||
"timezone": "Europe/Istanbul"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"farm_id": "boreas_enez_RES",
|
|
||||||
"name": "Boreas Enez RES",
|
|
||||||
"enabled": false,
|
|
||||||
"coordinates_file": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/coordinates/boreas_enez_RES_coordinates.json",
|
|
||||||
"distance_rings": [4000, 6000, 8000, 10000],
|
|
||||||
"ring_colors": ["#B71C1C", "#F94144", "#F8961E", "#90BE6D"],
|
|
||||||
"lightning_source_type": "api",
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "auto",
|
|
||||||
"padding_km": 5
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "manual",
|
|
||||||
"start_date": "08-12-2025",
|
|
||||||
"end_date": "23-03-2026"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/reports/boreas_enez_RES/",
|
|
||||||
"timezone": "Europe/Istanbul"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"farm_id": "maslaktepe_RES",
|
|
||||||
"name": "Maslaktepe RES",
|
|
||||||
"enabled": false,
|
|
||||||
"coordinates_file": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/coordinates/maslaktepe_RES_coordinates.json",
|
|
||||||
"distance_rings": [4000, 6000, 8000, 10000],
|
|
||||||
"ring_colors": ["#B71C1C", "#F94144", "#F8961E", "#90BE6D"],
|
|
||||||
"lightning_source_type": "api",
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "auto",
|
|
||||||
"padding_km": 5
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "manual",
|
|
||||||
"start_date": "08-12-2025",
|
|
||||||
"end_date": "23-03-2026"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/reports/maslaktepe_RES/",
|
|
||||||
"timezone": "Europe/Istanbul"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"farm_id": "Susurluk_RES",
|
|
||||||
"name": "Susurluk RES",
|
|
||||||
"enabled": false,
|
|
||||||
"coordinates_file": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/coordinates/susurluk_RES_coordinates.json",
|
|
||||||
"distance_rings": [1000, 2000, 3000, 4000, 10000],
|
|
||||||
"ring_colors": ["purple", "red", "orange", "coral", "green"],
|
|
||||||
"lightning_source_type": "api",
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "auto",
|
|
||||||
"padding_km": 5
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "manual",
|
|
||||||
"start_date": "05-11-2025",
|
|
||||||
"end_date": "08-12-2025"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/reports/susurluk_RES/",
|
|
||||||
"timezone": "Europe/Istanbul"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"farm_id": "benlikuyu_GES",
|
|
||||||
"name": "Benlikuyu GES",
|
|
||||||
"enabled": false,
|
|
||||||
"coordinates_file": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/coordinates/benlikuyu_GES_coordinates.json",
|
|
||||||
"distance_rings": [1000, 2000, 3000, 4000, 10000],
|
|
||||||
"ring_colors": ["purple", "red", "orange", "coral", "green"],
|
|
||||||
"lightning_source_type": "api",
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "auto",
|
|
||||||
"padding_km": 5
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "manual",
|
|
||||||
"start_date": "08-09-2025",
|
|
||||||
"end_date": "08-12-2025"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/reports/benlikuyu_GES/",
|
|
||||||
"timezone": "Europe/Istanbul"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"farm_id": "SOKE-01",
|
|
||||||
"name": "SOKE-01",
|
|
||||||
"enabled": false,
|
|
||||||
"coordinates_file": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/coordinates/SOKE-01.json",
|
|
||||||
"distance_rings": [1000, 2000, 4000, 8000, 10000],
|
|
||||||
"ring_colors": ["#D62828", "#F77F00", "#FCBF49", "#90BE6D", "#4D96FF"],
|
|
||||||
"lightning_source_type": "api",
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "auto",
|
|
||||||
"padding_km": 0.5
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "manual",
|
|
||||||
"start_date": "2026-01-22T07:00:00Z",
|
|
||||||
"end_date": "2026-01-22T08:00:00Z"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/reports/SOKE-01/",
|
|
||||||
"timezone": "Europe/Istanbul"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"farm_id": "Wind Farm Krnovo",
|
|
||||||
"name": "Wind Farm Krnovo",
|
|
||||||
"enabled": false,
|
|
||||||
"coordinates_file": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/coordinates/wind_farm_krnovo.json",
|
|
||||||
"distance_rings": [2000, 4000, 8000, 20000, 30000],
|
|
||||||
"ring_colors": ["#D62828", "#F77F00", "#FCBF49", "#90BE6D", "#4D96FF"],
|
|
||||||
"lightning_source_type": "csv",
|
|
||||||
"lightning_csv": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/lightnings/26_02_19_karadag.csv",
|
|
||||||
"api_params": {
|
|
||||||
"location_bounds": {
|
|
||||||
"method": "auto",
|
|
||||||
"padding_km": 0.5
|
|
||||||
},
|
|
||||||
"date_range": {
|
|
||||||
"method": "manual",
|
|
||||||
"start_date": "19-02-2026",
|
|
||||||
"end_date": "20-02-2026"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"report_config": {
|
|
||||||
"output_directory": "/Users/erdemerikci/Drive'ım/ERIKTRONIK/iklimco/Rapor/reports/wind_farm_krnovo/",
|
|
||||||
"timezone": "Europe/Podgorica"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
|
|
||||||
Loading…
x
Reference in New Issue
Block a user