← all branches

feat/atlas-scan

needs attentionviewing older commit
50e8a8c · incrementalpre-PRreviewed 2026-08-11 23:58 UTC3H · 5M · 7L · 3I
The branch
Purpose
Build an authenticated, private energy infrastructure atlas for Mexico — combining transmission (CENACE nodal prices), distribution (CFE GD capacity), and site intelligence (Power Finder) into a single internal decision tool for prospective data-center customers.
Goal
Add CFE distribution-grid capture pipeline (WFS + per-circuit hosting capacity) and integrate SCR (short-circuit ratio) into Power Finder ranking so MW load size actually changes which sites rank highest.
Sub-goals
  • SG-1: Authenticated /atlas route behind Supabase platform login
  • SG-2: Power Finder SCR/cociente-de-cortocircuito tier sort (this commit)
  • SG-3: CFE distribution-grid capture pipeline (enum → capacity → compare vs PF)
  • SG-4: Pilot analysis workflow (concordancia / cobertura / accionables lenses)
The changes (whole branch)
What
This commit adds SCR-based tier sorting to the Power Finder (network strength now primary sort key, savings secondary) and ships the full distribution-grid capture pipeline: 5 Python scripts that enumerate circuits by tiling the WFS, fill per-circuit hosting capacity, and cross-join the result against Power Finder substations by haversine distance. Includes a multi-agent Claude Code workflow for pilot analysis.
Why
Price-based sorting alone gives identical rankings for 10 MW and 300 MW loads; introducing SCR tier makes the finder sensitive to load size, surfacing Guaymas and Chihuahua over Los Mochis at 300 MW. The distribution pipeline adds an independent corroboration signal (two-source: CENACE transmission + CFE distribution) for site recommendations.
Areas
apps/platform/atlas-private+946232apps/platform/src+19511apps/web/src/app/(marketing)/atlas+750scripts/atlas/congestion+15230scripts/atlas/distribucion+5110scripts/atlas/pland+262414
Blast
122 files, +14444 / -59 lines across atlas private app, platform API routes, marketing page, and research scripts. All changes are either private/internal (atlas-private HTML, scripts/) or additive (new routes, new lenses). No breaking changes to existing platform APIs. Scripts are not deployed — local-only ETL.
analyze_pilot.workflow.js has hardcoded dev-machine absolute path — must be fixed before sharing Distribution capacity signal is GD generation hosting (not load capacity) — critical distinction documented in README and comments SQLite thread-safety latent bug in dist_capacity.py ThreadPoolExecutor — safe today, fragile tomorrow
ci· CI check API not accessible from runner (Bad credentials)coderabbit· no .coderabbit.yaml configured

Findings · 22

correctness5

high

SQLite connection shared across ThreadPoolExecutor workers

scripts/atlas/distribucion/dist_capacity.py:60

A single `con` is passed to all threads via concurrent `ex.map`. The write happens sequentially in the consumer loop (so no crash today), but this is an invisible invariant: any future `con.execute` inside `fetch_one` will hit `ProgrammingError` or silently corrupt the DB. Add `check_same_thread=False` + a threading.Lock, or open a connection per worker and collect results.

high

_pi global proxy counter has a race condition under ThreadPoolExecutor

scripts/atlas/distribucion/dist_common.py:49

`_opener()` reads and increments module-level `_pi` without a lock. With 4 workers calling `_opener()` concurrently, two threads can read the same value and pick the same proxy, defeating round-robin rotation. Fix: protect the read-modify-write with a `threading.Lock`.

medium

substations_of() fetches ALL divisions, filters in Python — silent data gap at scale

scripts/atlas/distribucion/dist_enumerate.py:23

`C.wfs('sigdis:subestacion')` passes no `cql_filter`; if the WFS paginates, target-division substations may be absent and the bbox will be silently wrong. Pass `cql=f"division='{div}'"` so the server-side filter is applied before any page cap.

medium

SCR tier conflates 'no data' (null) with measured-medium (SCR 5–20) — both tier 1

apps/platform/atlas-private/index.html:1694

A node with zero SCC data (unknown risk) sorts equal to a node confirmed at SCR 10 (moderate). For 300 MW loads, an unknown SCR is materially riskier than a measured one. Consider a separate `tierUnknown` (-1) that sorts after tier 0 above a MW threshold.

medium

disp fallback (tot - ins) overstates available hosting capacity

scripts/atlas/distribucion/dist_capacity.py:18

CFE's `capacidadDisponible` deducts committed loads and reserves; `tot - ins` is just the uninstalled portion — an upper bound. At minimum tag this field as 'estimated' or add a comment so downstream consumers don't treat it as the regulated figure.

security4

high

Hardcoded developer machine absolute path committed to repo

scripts/atlas/distribucion/analyze_pilot.workflow.js:10

`DIR = '/Users/alvaromigoya/Projects/batu-codebase/.claude/worktrees/atlas-scan/...'` — breaks on any other machine, leaks username and directory layout, and makes the workflow non-reproducible. Derive from `__dirname` / an `args` parameter, or accept `DIR` and `DATA` as workflow `args`.

low

File handle leak: json.dump(out, open(dst, 'w')) never closes the handle

scripts/atlas/distribucion/dist_compare.py:103

Use `with open(dst, 'w') as fh: json.dump(out, fh, ...)`. Same issue for `json.load(open(PF))` on line 61.

info

TLS certificate verification is default-on — positive finding

scripts/atlas/distribucion/dist_common.py

urllib.request is used without any ssl context override, so TLS validation is active. No action needed.

info

Proxy URL env var should be documented as trusted-operator-only input

scripts/atlas/distribucion/dist_common.py

DIST_PROXIES values are used verbatim in ProxyHandler. A comment in dist_common.py already explains the business isolation rationale; consider adding that the env var is trusted-operator-only.

conventions2

medium

Workflow CONTEXT strings embed the hardcoded DIR path — agents fail on other machines

scripts/atlas/distribucion/analyze_pilot.workflow.js:19

Every agent subprompt includes `${DATA}` which expands to the hardcoded absolute path. On another machine the agents are instructed to Read from a non-existent path, causing silent analysis failures. Fix flows automatically from fixing `DIR`.

low

Silent exception swallow in wfs() hides root cause on network failures

scripts/atlas/distribucion/dist_common.py:91

`except Exception: txt = ''` discards DNS failures, SSL errors, and timeouts — the script will spin through all retries with no diagnostic output. At minimum log `str(e)` before suppressing.

tests4

medium

SCR tier + sort logic is user-facing with no tests; scc=0 silently treated as no-data

apps/platform/atlas-private/index.html:1696

`scr=s.scc?s.scc/MW:null` — scc=0 is falsy in JS, so a node with zero short-circuit capacity gets tier=1 (middle) instead of tier=0 (weak). This can rank a genuinely weak node above a confirmed-weak one. MW=0 also produces NaN/null. A small Vitest or Node unit test on the two pure functions would catch these edge cases.

low

pf_key() classification has no unit tests; verde boundary is non-obvious

scripts/atlas/distribucion/dist_compare.py:39

atrap>=40 AND sat<=10 = verde — counterintuitive (high atrap + low sat). A node with atrap=39, sat=10 is amarillo, not verde. A 10-line doctest would make the boundary auditable.

low

disp fallback has no test for negative result (ins > tot)

scripts/atlas/distribucion/dist_capacity.py:27

If CFE data quality issues produce ins > tot, disp is negative — propagates into the DB and inflates 'holgura' counts in concordance tables.

info

Python ETL scripts without test framework — consistent with repo convention for atlas research tools

scripts/atlas/distribucion/dist_common.py

The congestion/ and pland/ scripts in the same directory follow the same no-test pattern. These are internal analyst tools, not production platform code. Acceptable for this tier.

improvement7

medium

substations_of() fetches entire national dataset on every division call

scripts/atlas/distribucion/dist_enumerate.py:23

At national scale (16 divisions) this sends 16 full-dataset WFS requests — the same load pattern that triggers the HTML-shell rate block documented in the code. Pass `cql_filter` and/or a bbox per division to reduce response size and block probability.

medium

Tier=1 default for missing SCC may mislead at large MW loads

apps/platform/atlas-private/index.html:1697

Showing 'red ajustada' (amber) for a node with no grid data is a reasonable UX hedge at 10 MW, but at 100–300 MW an unknown SCR deserves a distinct label ('sin datos de red') and lower priority. Consider a 4-tier scheme above a MW threshold.

low

capacidad() returns None for both 404 and transient failures — causes infinite retry

scripts/atlas/distribucion/dist_common.py:102

Circuit IDs that return HTTP 404 will always return None; they stay `cap_fetched_at IS NULL` and are retried on every run. Distinguish HTTP 404 (mark as 'NOT_FOUND') from transient errors to avoid wasted calls at scale.

low

No ETA or throughput reporting during tile enumeration

scripts/atlas/distribucion/dist_enumerate.py:98

With 6 s/tile sleep a national run could run for hours. Add `elapsed / i * (len(tl) - i)` to estimate remaining time, and log skipped tiles so operators can judge data completeness mid-run.

low

sys.argv parsed manually — no --help, crashes on missing value

scripts/atlas/distribucion/dist_capacity.py:80

All three pipeline scripts use manual `sys.argv` parsing: silently ignores unknown flags, crashes with IndexError if a flag lacks a value, no --help. A 10-line `argparse` replacement adds standard help docs and type safety.

low

Circuit centroid computed as mean of segment endpoints, not geographic centroid

scripts/atlas/distribucion/dist_enumerate.py:93

The mean of first-segment endpoints biases toward areas with denser segmentation near the substation. For the haversine join in dist_compare.py this may create false associations. Acceptable for pilot; document if used operationally.

low

Tile step 0.12° has no documented rationale or tuning guidance

scripts/atlas/distribucion/dist_enumerate.py:125

0.12° is ~13 km — adequate for northern Mexico but may trigger shell-blocks in dense metros (CDMX, MTY, GDL). Add a comment explaining the derivation and what symptom to look for when it needs tuning.

History · 16 commits

  1. 91aaedfsafeincremental0H · 1M · 1L2026-08-12 17:35
  2. cb8d915needs attentionincremental2H · 5M · 3L2026-08-12 17:29
  3. 4cbbe8aneeds attentionincremental0H · 3M · 8L2026-08-12 14:01
  4. 4aa3c02needs attentionincremental2H · 3M · 3L2026-08-12 02:35
  5. f5630b2needs attentionincremental0H · 4M · 5L2026-08-12 02:17
  6. 28fde5bneeds attentionincremental1H · 2M · 2L2026-08-12 01:53
  7. 0babe51needs attentionincremental1H · 2M · 5L2026-08-12 01:23
  8. 50e8a8cneeds attentionincremental3H · 5M · 7L2026-08-11 23:58current
  9. 3af4686needs attentionincremental3H · 5M · 5L2026-08-03 20:07
  10. f5d3266needs attentionincremental0H · 3M · 8L2026-08-03 19:40
  11. 9f8b61aneeds attentionincremental0H · 2M · 3L2026-08-03 19:15
  12. ea51fa0needs attentionincremental1H · 4M · 4L2026-08-03 19:02
  13. 2b33f2fneeds attentionincremental0H · 1M · 1L2026-07-18 05:09
  14. 2f8cf79needs attentionincremental1H · 3M · 4L2026-07-18 00:59
  15. 1616332safeincremental0H · 0M · 1L2026-07-18 00:22
  16. e997fd8needs attentionfull1H · 4M · 6L2026-07-17 23:53