← all branches

feat/atlas-scan

needs attentionviewing older commit
0babe51 · incrementalpre-PRreviewed 2026-08-12 01:23 UTC1H · 2M · 5L · 3I
The branch
Purpose
Build CFE distribution grid intelligence: scrape publicly accessible hosting-capacity data (WFS + circuit capacity API) to identify constrained vs available circuits across Mexico, cross-referenced with Power Finder substations.
Goal
Private atlas behind platform auth + CFE distribution grid capture pipeline (enumerate → capacity → compare) with an autonomous watcher that unblocks and runs the pilot hands-free.
Sub-goals
  • SG-1: Private /atlas route behind Batu platform login (authenticated edge rewrite)
  • SG-2: CFE distribution grid WFS client + SQLite pipeline (dist_common, dist_enumerate, dist_capacity, dist_compare)
  • SG-3: Optional rotating-proxy support for scaling beyond single-IP throttle
  • SG-4: Autonomous pilot watcher — polls WFS every 15min, runs full pipeline on unblock
The changes (whole branch)
What
Adds pilot_run.py — a single-file orchestrator that waits for the CFE WFS to become unblocked (polling every 15 min), then chains the three pipeline stages (enumerate → capacity → compare) for the Los Mochis corridor (División Noroeste, DB).
Why
The CFE WFS blocks by burst rate, not daily cap; an operator running the pipeline manually has to wait and re-try. This watcher automates the wait loop so the full pilot runs hands-free overnight.
Areas
scripts/atlas+133031apps/platform+3451apps/web+85257.gitignore+20
Blast
123 files, +14502/-59. Mostly scripts/atlas (83 files, research scripts + data). apps/platform adds the private /atlas route (15 files). No production SaaS logic changed.
research-script no-prod-impact external-api-polling
ci· no open PR — CI rollup unavailablecoderabbit· no .coderabbit.yaml configured

Findings · 11

correctness5

high

dist_enumerate.main() calls sys.exit() — silently aborts the full pipeline

scripts/atlas/distribucion/pilot_run.py:49

If no substations are found for the division, dist_enumerate.main() calls sys.exit() and the process dies. Stages 2/3 and 3/3 never run and 'PILOTO COMPLETO' is never printed. Consider wrapping in try/except SystemExit to emit a clear diagnostic and exit cleanly.

medium

Proxy presence skips WFS availability probe unconditionally

scripts/atlas/distribucion/pilot_run.py:41

When C._PROXIES is non-empty the poll loop is skipped entirely and the pipeline runs immediately — even if the WFS is still returning the shell HTML. dist_enumerate.main() would then operate on empty/malformed data. Consider running at least one availability probe even with proxies configured, or document the assumption explicitly.

medium

No per-stage error handling — a failure in stage N silently skips remaining stages

scripts/atlas/distribucion/pilot_run.py:49

dist_enumerate.main(), dist_capacity.main(), and dist_compare.main() are called sequentially with no try/except. If any stage raises, subsequent stages are skipped with no message. Wrap each call to at minimum print the stage name before re-raising.

low

Actual poll interval is longer than POLL_MIN due to probe execution time

scripts/atlas/distribucion/pilot_run.py:45

time.sleep(POLL_MIN * 60) is called after the probe completes. On slow connections the probe itself adds seconds. Negligible at 15-minute intervals, but the interval is drift-by-design rather than fixed.

low

C._PROXIES accesses a private/internal attribute

scripts/atlas/distribucion/pilot_run.py:41

If dist_common renames or restructures _PROXIES, the check silently evaluates to falsy and the poll loop always runs — even with proxies configured. Prefer a documented public constant or predicate in dist_common.

security2

low

Duplicate 'import dist_common as C' statement

scripts/atlas/distribucion/pilot_run.py:14

dist_common is imported twice (lines 13-14). Harmless in Python but is a copy-paste artifact. Remove the redundant import.

info

Broad except swallows unexpected errors in disponible()

scripts/atlas/distribucion/pilot_run.py:35

except Exception as e catches all non-ShellBlocked errors (including bugs like AttributeError) and treats them as 'blocked'. Intentional per the comment, but could hide bugs during development. Acceptable for a research script.

tests1

info

No tests — consistent with directory-wide pattern

scripts/atlas/distribucion/pilot_run.py

No tests exist for any sibling scripts in this directory. This is a research/operational script; the pattern is consistent. No action required.

improvement3

low

15-minute sleep is not interruptible — Ctrl-C unresponsive mid-wait

scripts/atlas/distribucion/pilot_run.py:41

time.sleep(POLL_MIN * 60) holds for up to 15 minutes. Splitting into 60 × 1-second sleeps or using threading.Event.wait() would let KeyboardInterrupt respond immediately without changing the effective poll frequency.

low

POLL_MIN unit conversion repeated in log message and sleep call

scripts/atlas/distribucion/pilot_run.py:45

POLL_MIN * 60 appears implicitly in both the sleep and the printed message. Naming poll_seconds = POLL_MIN * 60 at the top and using it in both places eliminates the silent unit conversion.

info

Pipeline stages called with positional args — fragile if signatures change

scripts/atlas/distribucion/pilot_run.py:49

dist_enumerate.main(DIV, STEP, BBOX) relies on positional order where the third param is bbox_override. Keyword args (bbox_override=BBOX) would survive reordering and clarify intent.

History · 16 commits

  1. 91aaedfsafeincremental0H · 1M · 1L2026-08-12 17:35
  2. cb8d915needs attentionincremental2H · 5M · 3L2026-08-12 17:29
  3. 4cbbe8aneeds attentionincremental0H · 3M · 8L2026-08-12 14:01
  4. 4aa3c02needs attentionincremental2H · 3M · 3L2026-08-12 02:35
  5. f5630b2needs attentionincremental0H · 4M · 5L2026-08-12 02:17
  6. 28fde5bneeds attentionincremental1H · 2M · 2L2026-08-12 01:53
  7. 0babe51needs attentionincremental1H · 2M · 5L2026-08-12 01:23current
  8. 50e8a8cneeds attentionincremental3H · 5M · 7L2026-08-11 23:58
  9. 3af4686needs attentionincremental3H · 5M · 5L2026-08-03 20:07
  10. f5d3266needs attentionincremental0H · 3M · 8L2026-08-03 19:40
  11. 9f8b61aneeds attentionincremental0H · 2M · 3L2026-08-03 19:15
  12. ea51fa0needs attentionincremental1H · 4M · 4L2026-08-03 19:02
  13. 2b33f2fneeds attentionincremental0H · 1M · 1L2026-07-18 05:09
  14. 2f8cf79needs attentionincremental1H · 3M · 4L2026-07-18 00:59
  15. 1616332safeincremental0H · 0M · 1L2026-07-18 00:22
  16. e997fd8needs attentionfull1H · 4M · 6L2026-07-17 23:53