8 Commits

Author SHA1 Message Date
TheFurya 90399f41b7 Merge branch 'develop' into main 2026-02-11 12:00:31 +01:00
Julian Tabel aadd730002 Mark PokeDB import feature and subtasks as completed
CI / backend-lint (push) Successful in 11s
CI / frontend-lint (push) Successful in 30s
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 11:56:25 +01:00
Julian Tabel 872d7872ce Validate and regenerate all seed data from PokeDB
- Regenerate seed JSON for all 37 games with more complete PokeDB data
- Add category field to games.json (original/enhanced/remake/sequel/spinoff)
- Include all 1350 pokemon in pokemon.json with types and local sprites
- Build reverse index for PokeDB form lookups (types/sprites for evolutions)
- Move sprites to frontend/public/sprites, reference as /sprites/{id}.webp
- Truncate Sw/Sh den names to fit DB VARCHAR(100) limit
- Deduplicate route names and merge unnamed child areas into parent routes
- Populate 7 previously empty games (Sw/Sh, BDSP, PLA, Sc/Vi)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 11:52:51 +01:00
Julian Tabel df55233c62 Add seed JSON output (per-game, games.json, pokemon.json)
Wire output module into CLI pipeline: route ordering, special encounter
merging, and JSON writing for per-game encounters, global games list,
and pokemon list with types and sprite paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:59:56 +01:00
Julian Tabel 29b954726a Add PokeDB sprite downloading (100x100 WebP)
Download pokemon sprites from PokeDB CDN during import, cached locally
as {pokeapi_id}.webp. Replaces PokeAPI GitHub sprite URLs. ~4.6MB for
all 1119 unique sprites.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:24:43 +01:00
Julian Tabel d80c59047c Add core encounter processing pipeline
Filter by game version, parse levels and rate variants across all
generations, aggregate encounters by pokemon+method, and build
parent/child route hierarchy. Also completes encounter method coverage
(73/73) and pokemon form mapping (1180/1181) with manual overrides.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:12:55 +01:00
Julian Tabel df7ea64b9e Add reference data mappings and auto-download for PokeDB import tool
Add mappings module with pokemon form, location area, encounter method,
and version mappings. Auto-download PokeDB JSON exports from CDN on
first run, caching in .pokedb_cache/.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 10:02:57 +01:00
Julian Tabel 1aa67665ff Add Python tool scaffold for PokeDB data import
Set up tools/import-pokedb/ with CLI, JSON loader, and output models.
Replaces the Go/PokeAPI approach with local PokeDB.org JSON processing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-02-11 09:49:51 +01:00
55 changed files with 465450 additions and 129817 deletions
@@ -1,17 +1,22 @@
--- ---
# nuzlocke-tracker-bs05 # nuzlocke-tracker-bs05
title: Build PokeDB.org data import tool title: Build PokeDB.org data import tool
status: draft status: completed
type: task type: feature
priority: normal priority: normal
created_at: 2026-02-10T14:04:11Z created_at: 2026-02-10T14:04:11Z
updated_at: 2026-02-10T14:31:08Z updated_at: 2026-02-11T10:54:04Z
parent: nuzlocke-tracker-rzu4 parent: nuzlocke-tracker-rzu4
blocking: blocking:
- nuzlocke-tracker-spx3 - nuzlocke-tracker-spx3
--- ---
Build a Go tool that converts PokeDB.org's JSON data export into our existing seed JSON format. This replaces PokeAPI as the single source of truth for ALL games (Gen 1-9). Build a standalone Python tool that converts PokeDB.org's JSON data export into our existing seed JSON format. This replaces PokeAPI as the single source of truth for ALL games (Gen 1-9).
Python was chosen over Go because:
- The backend is already Python, so the team is familiar with it
- We're processing local JSON files — no need for Go's concurrency
- Remains a standalone tool in `tools/import-pokedb/`, not part of the backend
## Data source ## Data source
@@ -64,26 +69,15 @@ Each encounter record has:
- `visible` — overworld vs hidden encounter - `visible` — overworld vs hidden encounter
- Max Raid and Tera Raid fields for special encounters - Max Raid and Tera Raid fields for special encounters
## Implementation approach ## Subtasks
### Checklist Work is broken into child task beans:
- [ ] Set up project structure in `tools/import-pokedb/`
- [ ] Download and cache PokeDB JSON export files - [ ] **Set up Python tool scaffold** — project structure, CLI entry point, PokeDB JSON file loading
- [ ] Parse PokeDB encounters, locations, location_areas, versions, pokemon_forms - [ ] **Build reference data mappings** — pokemon_form → pokeapi_id, location_area → name/region, encounter method mapping
- [ ] Build lookup maps: pokemon_form_identifier → pokeapi_id (using existing `pokemon.json`) - [ ] **Core encounter processing** — filter by game version, parse levels, handle rate variants, group by location area
- [ ] Build lookup maps: location_area_identifier → location name + region - [ ] **Output seed JSON** — produce per-game JSON in existing format, integrate route ordering + special encounters
- [ ] Filter encounters by target game version - [ ] **Validation & full generation** — compare against existing data, run for all games, fix discrepancies
- [ ] Map PokeDB encounter methods to our seed format methods (73 → simplified set)
- [ ] Parse level strings ("2 - 4" → min_level: 2, max_level: 4)
- [ ] Handle rate variants per game generation:
- For now, flatten time/weather/season rates into `encounter_rate` (use the max or average)
- Preserve raw variant data for future use (see nuzlocke-tracker-oqfo)
- [ ] Group encounters by location area → route output
- [ ] Apply route ordering (use existing route_order.json or generate from location data)
- [ ] Output in existing `{game}.json` seed format
- [ ] Generate seed data for ALL games, replacing PokeAPI as the single source of truth
- [ ] Compare output against existing PokeAPI-sourced data to validate accuracy
- [ ] Run for all games and verify output
## Encounter method mapping (draft) ## Encounter method mapping (draft)
@@ -0,0 +1,30 @@
---
# nuzlocke-tracker-dqyb
title: Set up Python tool scaffold
status: completed
type: task
priority: normal
created_at: 2026-02-11T08:42:58Z
updated_at: 2026-02-11T08:49:55Z
parent: nuzlocke-tracker-bs05
blocking:
- nuzlocke-tracker-zno2
---
Set up the standalone Python tool project in `tools/import-pokedb/`.
## Checklist
- [x] Create `tools/import-pokedb/` directory structure
- [x] Set up `pyproject.toml` with dependencies (just stdlib should suffice for JSON processing, maybe `click` for CLI)
- [x] Create CLI entry point (`__main__.py` or similar) that accepts:
- Path to directory containing PokeDB JSON export files
- Target output directory (default: `backend/src/app/seeds/data/`)
- Optional: specific game version to generate (default: all)
- [x] Load and parse all PokeDB JSON files: `encounters.json`, `locations.json`, `location_areas.json`, `encounter_methods.json`, `versions.json`, `pokemon_forms.json`
- [x] Basic validation that all expected files are present and parseable
## Notes
- Keep it as a standalone tool, not part of the backend
- The PokeDB JSON files are downloaded manually from https://pokedb.org/data-export — no need to automate the download
- Model the CLI similarly to how `tools/fetch-pokeapi/` works (cd into dir, run the tool)
@@ -0,0 +1,31 @@
---
# nuzlocke-tracker-gkcy
title: Output seed JSON
status: completed
type: task
priority: normal
created_at: 2026-02-11T08:43:21Z
updated_at: 2026-02-11T10:00:00Z
parent: nuzlocke-tracker-bs05
blocking:
- nuzlocke-tracker-vdks
---
Generate the final per-game JSON files in the existing seed format.
## Checklist
- [x] **Apply route ordering**: Use the existing `backend/src/app/seeds/route_order.json` to assign `order` values to routes. Handle aliases (e.g. "red-blue" → "firered-leafgreen"). Log warnings for routes not in the order file.
- [x] **Merge special encounters**: Integrate starters, gifts, fossils, and trades from `backend/src/app/seeds/special_encounters.json` into the appropriate routes. Pokemon names are resolved to proper display names via PokemonMapper.
- [x] **Output per-game JSON**: Write `{game-slug}.json` files matching the existing format:
```json
[{"name": "Route 1", "order": 3, "encounters": [...], "children": []}]
```
- [x] **Output games.json**: Generate the global games list from `version_groups.json` — 38 games written, matching existing count.
- [x] **Output pokemon.json**: Generate the global pokemon list including all pokemon referenced in any encounter. Include pokeapi_id, national_dex, name, types, sprite_url.
- [x] **Handle version exclusives**: Encounters are filtered by `version_identifiers` per game — verified FireRed vs LeafGreen have 18 exclusives each.
## Notes
- The output must be a drop-in replacement for the existing files in `backend/src/app/seeds/data/`
- Boss data (`{game}-bosses.json`) is NOT generated by this tool — it's manually curated
- Evolutions data is also separate (currently from PokeAPI) — out of scope for this task
@@ -0,0 +1,34 @@
---
# nuzlocke-tracker-rfg0
title: Core encounter processing
status: completed
type: task
priority: normal
created_at: 2026-02-11T08:43:12Z
updated_at: 2026-02-11T09:12:59Z
parent: nuzlocke-tracker-bs05
blocking:
- nuzlocke-tracker-gkcy
---
Implement the core logic that transforms raw PokeDB encounter records into our internal format.
## Checklist
- [x] **Filter by game version**: Given a target game slug, select only encounters where `version_identifiers` includes that game
- [x] **Parse level strings**: Convert "2 - 4" → min_level=2, max_level=4; "67" → min_level=67, max_level=67
- [x] **Handle rate variants per generation**:
- Gen 1/3/6: use `rate_overall` directly as `encounter_rate`
- Gen 2/4: `rate_morning`, `rate_day`, `rate_night` — flatten to max or average for `encounter_rate`
- Gen 5: `rate_spring` through `rate_winter` — flatten similarly
- Gen 8 Sw/Sh: `weather_*_rate` fields — flatten to max
- Gen 8 Legends Arceus: `during_*` / `while_*` booleans — convert to a presence-based rate
- Gen 9 Sc/Vi: `probability_*` fields (spawn weights, not percentages) — normalize to percentages
- Preserve raw variant data in a way that nuzlocke-tracker-oqfo can use later
- [x] **Aggregate encounters**: Group by (pokemon, method, location_area) and merge level ranges / rates where appropriate (same logic as the Go tool's aggregation)
- [x] **Group by location area**: Collect all encounters for a location area into a route structure
- [x] **Handle parent/child routes**: Multi-area locations (e.g. Safari Zone) should produce parent routes with children, matching the existing hierarchical format
## Notes
- Rate parsing needs to handle percentage strings like "40%" as well as bare numbers
- The Go tool aggregates encounters with the same pokemon+method at a location into a single entry with merged level ranges — replicate this
@@ -0,0 +1,26 @@
---
# nuzlocke-tracker-vdks
title: Validation and full generation
status: completed
type: task
priority: normal
created_at: 2026-02-11T08:43:29Z
updated_at: 2026-02-11T10:52:55Z
parent: nuzlocke-tracker-bs05
---
Validate the new tool's output against existing data and generate seed data for all games.
## Checklist
- [x] **Diff against existing data**: Compared all 37 games. Route names differ systematically (PokeDB uses proper punctuation/region qualifiers). Encounter counts are generally higher (PokeDB more complete). Version exclusives verified.
- [x] **Fix discrepancies**: Fixed: games.json missing `category` field, pokemon.json missing 231 non-encountered pokemon (evolutions etc.), pokemon name casing in special encounters, sprite coverage for all 1350 pokemon, type coverage using fallback to existing data.
- [x] **Generate for all games**: All 37 games (+ legends-z-a with no data) generated successfully. All 39 JSON files validated. 7 previously empty games now populated (Sw/Sh, BDSP, PLA, Sc/Vi).
- [x] **New game coverage**: Sword/Shield (1052 routes, 8900 encounters), Scarlet/Violet (415 routes, ~3850 encounters), Legends Arceus (88 routes, 1582 encounters), BDSP (177 routes, ~1840 encounters) — all reasonable.
- [ ] **Update route_order.json**: 11 version groups have route orders, 11 are missing (diamond-pearl, black-white, black-2-white-2, x-y, sun-moon, ultra-sun-ultra-moon, sword-shield, brilliant-diamond-shining-pearl, legends-arceus, scarlet-violet). Requires manual curation.
- [ ] **Update special_encounters.json**: Currently only covers 3 version groups (firered-leafgreen, heartgold-soulsilver, emerald + aliases). Requires manual curation for other games.
## Notes
- This is the final validation step before we can replace PokeAPI as the data source
- Some discrepancies are expected — PokeDB may have more complete data than PokeAPI
- Route ordering for new games will likely need manual work
@@ -0,0 +1,26 @@
---
# nuzlocke-tracker-zno2
title: Build reference data mappings
status: completed
type: task
priority: normal
created_at: 2026-02-11T08:43:02Z
updated_at: 2026-02-11T09:03:01Z
parent: nuzlocke-tracker-bs05
blocking:
- nuzlocke-tracker-rfg0
---
Build the lookup maps needed to translate PokeDB identifiers into our seed format.
## Checklist
- [x] **Pokemon form mapping**: Map `pokemon_form_identifier` (e.g. "pidgey-default", "mr-mime-default") to `pokeapi_id` using the existing `backend/src/app/seeds/data/pokemon.json` as reference. Handle naming convention differences between PokeDB and PokeAPI (may need fuzzy matching or a manual override table).
- [x] **Location area mapping**: Map `location_area_identifier` to human-readable location names and regions using `locations.json` and `location_areas.json`. Produce names matching our existing format (e.g. "Route 1", "Viridian Forest").
- [x] **Encounter method mapping**: Map PokeDB's 73 encounter methods to our simplified set. See the draft mapping in the parent bean. Implement as a dictionary/config that's easy to extend.
- [x] **Version mapping**: Map PokeDB `version_identifiers` to our game slugs (should mostly be 1:1 but verify).
## Notes
- The pokemon form mapping is the trickiest part — PokeDB uses identifiers like "mr-mime-default" while our pokemon.json uses names like "Mr. Mime" and pokeapi IDs
- Log warnings for any unmapped identifiers so we can add overrides
- The `pokemon_forms.json` from PokeDB may help bridge the gap
+5 -1
View File
@@ -60,8 +60,12 @@ temp/
*.tmp *.tmp
*.temp *.temp
# PokeAPI fetch cache # PokeAPI / PokeDB data cache
.pokeapi_cache/ .pokeapi_cache/
.pokedb_cache/
# Generated sprites (downloaded by import-pokedb tool)
frontend/public/sprites/
# Go build output # Go build output
tools/fetch-pokeapi/fetch-pokeapi tools/fetch-pokeapi/fetch-pokeapi
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
+326 -326
View File
@@ -1,344 +1,344 @@
[ [
{ {
"name": "Pokemon Alpha Sapphire", "name": "Pokemon Red",
"slug": "alpha-sapphire", "slug": "red",
"generation": 6, "generation": 1,
"region": "hoenn", "region": "kanto",
"category": "remake", "release_year": 1996,
"release_year": 2014, "color": "#FF1111",
"color": "#26649C" "category": "original"
},
{
"name": "Pokemon Black",
"slug": "black",
"generation": 5,
"region": "unova",
"category": "original",
"release_year": 2010,
"color": "#444444"
},
{
"name": "Pokemon Black 2",
"slug": "black-2",
"generation": 5,
"region": "unova",
"category": "sequel",
"release_year": 2012,
"color": "#424B50"
}, },
{ {
"name": "Pokemon Blue", "name": "Pokemon Blue",
"slug": "blue", "slug": "blue",
"generation": 1, "generation": 1,
"region": "kanto", "region": "kanto",
"category": "original",
"release_year": 1996, "release_year": 1996,
"color": "#1111FF" "color": "#1111FF",
}, "category": "original"
{
"name": "Pokemon Brilliant Diamond",
"slug": "brilliant-diamond",
"generation": 8,
"region": "sinnoh",
"category": "remake",
"release_year": 2021,
"color": "#44BAE5"
},
{
"name": "Pokemon Crystal",
"slug": "crystal",
"generation": 2,
"region": "johto",
"category": "enhanced",
"release_year": 2000,
"color": "#4FD9FF"
},
{
"name": "Pokemon Diamond",
"slug": "diamond",
"generation": 4,
"region": "sinnoh",
"category": "original",
"release_year": 2006,
"color": "#AAAAFF"
},
{
"name": "Pokemon Emerald",
"slug": "emerald",
"generation": 3,
"region": "hoenn",
"category": "enhanced",
"release_year": 2005,
"color": "#00A000"
},
{
"name": "Pokemon FireRed",
"slug": "firered",
"generation": 3,
"region": "kanto",
"category": "remake",
"release_year": 2004,
"color": "#FF7327"
},
{
"name": "Pokemon Gold",
"slug": "gold",
"generation": 2,
"region": "johto",
"category": "original",
"release_year": 1999,
"color": "#DAA520"
},
{
"name": "Pokemon HeartGold",
"slug": "heartgold",
"generation": 4,
"region": "johto",
"category": "remake",
"release_year": 2010,
"color": "#B69E00"
},
{
"name": "Pokemon LeafGreen",
"slug": "leafgreen",
"generation": 3,
"region": "kanto",
"category": "remake",
"release_year": 2004,
"color": "#00DD00"
},
{
"name": "Pokemon Legends: Arceus",
"slug": "legends-arceus",
"generation": 8,
"region": "hisui",
"category": "spinoff",
"release_year": 2022,
"color": "#36597B"
},
{
"name": "Pokemon Legends: Z-A",
"slug": "legends-z-a",
"generation": 9,
"region": "lumiose",
"category": "spinoff",
"release_year": 2025,
"color": "#3A7BDB"
},
{
"name": "Pokemon Let's Go Eevee",
"slug": "lets-go-eevee",
"generation": 7,
"region": "kanto",
"category": "remake",
"release_year": 2018,
"color": "#D4924B"
},
{
"name": "Pokemon Let's Go Pikachu",
"slug": "lets-go-pikachu",
"generation": 7,
"region": "kanto",
"category": "remake",
"release_year": 2018,
"color": "#F5DA00"
},
{
"name": "Pokemon Moon",
"slug": "moon",
"generation": 7,
"region": "alola",
"category": "original",
"release_year": 2016,
"color": "#5599CA"
},
{
"name": "Pokemon Omega Ruby",
"slug": "omega-ruby",
"generation": 6,
"region": "hoenn",
"category": "remake",
"release_year": 2014,
"color": "#CF3025"
},
{
"name": "Pokemon Pearl",
"slug": "pearl",
"generation": 4,
"region": "sinnoh",
"category": "original",
"release_year": 2006,
"color": "#FFAAAA"
},
{
"name": "Pokemon Platinum",
"slug": "platinum",
"generation": 4,
"region": "sinnoh",
"category": "enhanced",
"release_year": 2008,
"color": "#999999"
},
{
"name": "Pokemon Red",
"slug": "red",
"generation": 1,
"region": "kanto",
"category": "original",
"release_year": 1996,
"color": "#FF1111"
},
{
"name": "Pokemon Ruby",
"slug": "ruby",
"generation": 3,
"region": "hoenn",
"category": "original",
"release_year": 2002,
"color": "#A00000"
},
{
"name": "Pokemon Sapphire",
"slug": "sapphire",
"generation": 3,
"region": "hoenn",
"category": "original",
"release_year": 2002,
"color": "#0000A0"
},
{
"name": "Pokemon Scarlet",
"slug": "scarlet",
"generation": 9,
"region": "paldea",
"category": "original",
"release_year": 2022,
"color": "#F93C3C"
},
{
"name": "Pokemon Shield",
"slug": "shield",
"generation": 8,
"region": "galar",
"category": "original",
"release_year": 2019,
"color": "#EF3B6E"
},
{
"name": "Pokemon Shining Pearl",
"slug": "shining-pearl",
"generation": 8,
"region": "sinnoh",
"category": "remake",
"release_year": 2021,
"color": "#E18AAA"
},
{
"name": "Pokemon Silver",
"slug": "silver",
"generation": 2,
"region": "johto",
"category": "original",
"release_year": 1999,
"color": "#C0C0C0"
},
{
"name": "Pokemon SoulSilver",
"slug": "soulsilver",
"generation": 4,
"region": "johto",
"category": "remake",
"release_year": 2010,
"color": "#C0C0E0"
},
{
"name": "Pokemon Sun",
"slug": "sun",
"generation": 7,
"region": "alola",
"category": "original",
"release_year": 2016,
"color": "#F1912B"
},
{
"name": "Pokemon Sword",
"slug": "sword",
"generation": 8,
"region": "galar",
"category": "original",
"release_year": 2019,
"color": "#00D4E7"
},
{
"name": "Pokemon Ultra Moon",
"slug": "ultra-moon",
"generation": 7,
"region": "alola",
"category": "enhanced",
"release_year": 2017,
"color": "#204E8C"
},
{
"name": "Pokemon Ultra Sun",
"slug": "ultra-sun",
"generation": 7,
"region": "alola",
"category": "enhanced",
"release_year": 2017,
"color": "#E95B2B"
},
{
"name": "Pokemon Violet",
"slug": "violet",
"generation": 9,
"region": "paldea",
"category": "original",
"release_year": 2022,
"color": "#A96EEC"
},
{
"name": "Pokemon White",
"slug": "white",
"generation": 5,
"region": "unova",
"category": "original",
"release_year": 2010,
"color": "#E1E1E1"
},
{
"name": "Pokemon White 2",
"slug": "white-2",
"generation": 5,
"region": "unova",
"category": "sequel",
"release_year": 2012,
"color": "#E3CED0"
},
{
"name": "Pokemon X",
"slug": "x",
"generation": 6,
"region": "kalos",
"category": "original",
"release_year": 2013,
"color": "#025DA6"
},
{
"name": "Pokemon Y",
"slug": "y",
"generation": 6,
"region": "kalos",
"category": "original",
"release_year": 2013,
"color": "#EA1A3E"
}, },
{ {
"name": "Pokemon Yellow", "name": "Pokemon Yellow",
"slug": "yellow", "slug": "yellow",
"generation": 1, "generation": 1,
"region": "kanto", "region": "kanto",
"category": "enhanced",
"release_year": 1998, "release_year": 1998,
"color": "#FFD733" "color": "#FFD733",
"category": "enhanced"
},
{
"name": "Pokemon Gold",
"slug": "gold",
"generation": 2,
"region": "johto",
"release_year": 1999,
"color": "#DAA520",
"category": "original"
},
{
"name": "Pokemon Silver",
"slug": "silver",
"generation": 2,
"region": "johto",
"release_year": 1999,
"color": "#C0C0C0",
"category": "original"
},
{
"name": "Pokemon Crystal",
"slug": "crystal",
"generation": 2,
"region": "johto",
"release_year": 2000,
"color": "#4FD9FF",
"category": "enhanced"
},
{
"name": "Pokemon Ruby",
"slug": "ruby",
"generation": 3,
"region": "hoenn",
"release_year": 2002,
"color": "#A00000",
"category": "original"
},
{
"name": "Pokemon Sapphire",
"slug": "sapphire",
"generation": 3,
"region": "hoenn",
"release_year": 2002,
"color": "#0000A0",
"category": "original"
},
{
"name": "Pokemon Emerald",
"slug": "emerald",
"generation": 3,
"region": "hoenn",
"release_year": 2005,
"color": "#00A000",
"category": "enhanced"
},
{
"name": "Pokemon FireRed",
"slug": "firered",
"generation": 3,
"region": "kanto",
"release_year": 2004,
"color": "#FF7327",
"category": "remake"
},
{
"name": "Pokemon LeafGreen",
"slug": "leafgreen",
"generation": 3,
"region": "kanto",
"release_year": 2004,
"color": "#00DD00",
"category": "remake"
},
{
"name": "Pokemon Diamond",
"slug": "diamond",
"generation": 4,
"region": "sinnoh",
"release_year": 2006,
"color": "#AAAAFF",
"category": "original"
},
{
"name": "Pokemon Pearl",
"slug": "pearl",
"generation": 4,
"region": "sinnoh",
"release_year": 2006,
"color": "#FFAAAA",
"category": "original"
},
{
"name": "Pokemon Platinum",
"slug": "platinum",
"generation": 4,
"region": "sinnoh",
"release_year": 2008,
"color": "#999999",
"category": "enhanced"
},
{
"name": "Pokemon HeartGold",
"slug": "heartgold",
"generation": 4,
"region": "johto",
"release_year": 2010,
"color": "#B69E00",
"category": "remake"
},
{
"name": "Pokemon SoulSilver",
"slug": "soulsilver",
"generation": 4,
"region": "johto",
"release_year": 2010,
"color": "#C0C0E0",
"category": "remake"
},
{
"name": "Pokemon Black",
"slug": "black",
"generation": 5,
"region": "unova",
"release_year": 2010,
"color": "#444444",
"category": "original"
},
{
"name": "Pokemon White",
"slug": "white",
"generation": 5,
"region": "unova",
"release_year": 2010,
"color": "#E1E1E1",
"category": "original"
},
{
"name": "Pokemon Black 2",
"slug": "black-2",
"generation": 5,
"region": "unova",
"release_year": 2012,
"color": "#424B50",
"category": "sequel"
},
{
"name": "Pokemon White 2",
"slug": "white-2",
"generation": 5,
"region": "unova",
"release_year": 2012,
"color": "#E3CED0",
"category": "sequel"
},
{
"name": "Pokemon X",
"slug": "x",
"generation": 6,
"region": "kalos",
"release_year": 2013,
"color": "#025DA6",
"category": "original"
},
{
"name": "Pokemon Y",
"slug": "y",
"generation": 6,
"region": "kalos",
"release_year": 2013,
"color": "#EA1A3E",
"category": "original"
},
{
"name": "Pokemon Omega Ruby",
"slug": "omega-ruby",
"generation": 6,
"region": "hoenn",
"release_year": 2014,
"color": "#CF3025",
"category": "remake"
},
{
"name": "Pokemon Alpha Sapphire",
"slug": "alpha-sapphire",
"generation": 6,
"region": "hoenn",
"release_year": 2014,
"color": "#26649C",
"category": "remake"
},
{
"name": "Pokemon Sun",
"slug": "sun",
"generation": 7,
"region": "alola",
"release_year": 2016,
"color": "#F1912B",
"category": "original"
},
{
"name": "Pokemon Moon",
"slug": "moon",
"generation": 7,
"region": "alola",
"release_year": 2016,
"color": "#5599CA",
"category": "original"
},
{
"name": "Pokemon Ultra Sun",
"slug": "ultra-sun",
"generation": 7,
"region": "alola",
"release_year": 2017,
"color": "#E95B2B",
"category": "enhanced"
},
{
"name": "Pokemon Ultra Moon",
"slug": "ultra-moon",
"generation": 7,
"region": "alola",
"release_year": 2017,
"color": "#204E8C",
"category": "enhanced"
},
{
"name": "Pokemon Let's Go Pikachu",
"slug": "lets-go-pikachu",
"generation": 7,
"region": "kanto",
"release_year": 2018,
"color": "#F5DA00",
"category": "remake"
},
{
"name": "Pokemon Let's Go Eevee",
"slug": "lets-go-eevee",
"generation": 7,
"region": "kanto",
"release_year": 2018,
"color": "#D4924B",
"category": "remake"
},
{
"name": "Pokemon Sword",
"slug": "sword",
"generation": 8,
"region": "galar",
"release_year": 2019,
"color": "#00D4E7",
"category": "original"
},
{
"name": "Pokemon Shield",
"slug": "shield",
"generation": 8,
"region": "galar",
"release_year": 2019,
"color": "#EF3B6E",
"category": "original"
},
{
"name": "Pokemon Brilliant Diamond",
"slug": "brilliant-diamond",
"generation": 8,
"region": "sinnoh",
"release_year": 2021,
"color": "#44BAE5",
"category": "remake"
},
{
"name": "Pokemon Shining Pearl",
"slug": "shining-pearl",
"generation": 8,
"region": "sinnoh",
"release_year": 2021,
"color": "#E18AAA",
"category": "remake"
},
{
"name": "Pokemon Legends: Arceus",
"slug": "legends-arceus",
"generation": 8,
"region": "hisui",
"release_year": 2022,
"color": "#36597B",
"category": "spinoff"
},
{
"name": "Pokemon Scarlet",
"slug": "scarlet",
"generation": 9,
"region": "paldea",
"release_year": 2022,
"color": "#F93C3C",
"category": "original"
},
{
"name": "Pokemon Violet",
"slug": "violet",
"generation": 9,
"region": "paldea",
"release_year": 2022,
"color": "#A96EEC",
"category": "original"
},
{
"name": "Pokemon Legends: Z-A",
"slug": "legends-z-a",
"generation": 9,
"region": "lumiose",
"release_year": 2025,
"color": "#3A7BDB",
"category": "spinoff"
} }
] ]
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,239 @@
"""CLI entry point for the PokeDB import tool.
Usage:
# From tools/import-pokedb/ (auto-downloads PokeDB data on first run):
python -m import_pokedb
# With options:
python -m import_pokedb --game firered
python -m import_pokedb --pokedb-dir ~/my-pokedb-data/
python -m import_pokedb --output /tmp/seed-output/
"""
from __future__ import annotations
import argparse
import sys
from pathlib import Path
from .loader import load_pokedb_data, load_seed_config
from .mappings import PokemonMapper, LocationMapper, build_version_map, map_encounter_method
from .output import sort_routes, merge_special_encounters, write_game_json, write_games_json, write_pokemon_json
from .processing import filter_encounters_for_game, process_encounters, build_routes
from .sprites import download_all_sprites, download_sprites
SEEDS_DIR_CANDIDATES = [
Path("backend/src/app/seeds"), # from repo root
Path("../../backend/src/app/seeds"), # from tools/import-pokedb/
]
SPRITES_DIR_CANDIDATES = [
Path("frontend/public/sprites"), # from repo root
Path("../../frontend/public/sprites"), # from tools/import-pokedb/
]
def find_seeds_dir() -> Path:
"""Locate the backend seeds directory."""
for candidate in SEEDS_DIR_CANDIDATES:
if (candidate / "version_groups.json").exists():
return candidate.resolve()
# Fallback
return Path("backend/src/app/seeds").resolve()
def find_sprites_dir() -> Path:
"""Locate the frontend sprites directory."""
for candidate in SPRITES_DIR_CANDIDATES:
if candidate.parent.exists():
return candidate.resolve()
# Fallback
return Path("frontend/public/sprites").resolve()
def build_parser() -> argparse.ArgumentParser:
parser = argparse.ArgumentParser(
prog="import-pokedb",
description="Convert PokeDB.org JSON data exports into nuzlocke-tracker seed format.",
)
parser.add_argument(
"--pokedb-dir",
type=Path,
default=None,
help="Path to directory containing PokeDB JSON export files (default: .pokedb_cache/)",
)
parser.add_argument(
"--output",
type=Path,
default=None,
help="Output directory for seed JSON files (default: backend/src/app/seeds/data/)",
)
parser.add_argument(
"--game",
type=str,
default=None,
help="Generate data for a specific game slug only (default: all games)",
)
return parser
def main(argv: list[str] | None = None) -> None:
parser = build_parser()
args = parser.parse_args(argv)
seeds_dir = find_seeds_dir()
pokedb_dir: Path = args.pokedb_dir or (seeds_dir / ".pokedb_cache")
output_dir: Path = args.output or (seeds_dir / "data")
output_dir.mkdir(parents=True, exist_ok=True)
print(f"PokeDB data: {pokedb_dir.resolve()}")
print(f"Seeds config: {seeds_dir}")
print(f"Output: {output_dir.resolve()}")
print()
# Load PokeDB export data
pokedb = load_pokedb_data(pokedb_dir)
print(pokedb.summary())
print()
# Load existing seed configuration
config = load_seed_config(seeds_dir)
print(f"Loaded {len(config.version_groups)} version groups")
print(f"Loaded route order for {len(config.route_order)} version groups")
if config.special_encounters:
se_count = len(config.special_encounters.get("encounters", {}))
print(f"Loaded special encounters for {se_count} version groups")
print()
# Determine which games to process
target_game = args.game
if target_game:
found = False
for vg_info in config.version_groups.values():
if target_game in vg_info.get("versions", []):
found = True
break
if not found:
print(f"Error: Game '{target_game}' not found in version_groups.json", file=sys.stderr)
sys.exit(1)
print(f"Target: {target_game}")
else:
total_games = sum(
len(vg.get("versions", []))
for vg in config.version_groups.values()
)
print(f"Target: all {total_games} games")
# Build mappings
print("\nBuilding mappings...")
pokemon_json = seeds_dir / "data" / "pokemon.json"
pokemon_mapper = PokemonMapper(pokemon_json, pokedb)
location_mapper = LocationMapper(pokedb)
version_map = build_version_map(pokedb, config.version_groups)
print(f" Mapped {len(version_map)} PokeDB versions to our game slugs")
# Report encounter method coverage
pokedb_methods = {e.get("encounter_method_identifier", "") for e in pokedb.encounters}
pokedb_methods.discard("")
mapped_methods = {m for m in pokedb_methods if map_encounter_method(m) is not None}
unmapped_methods = pokedb_methods - mapped_methods
print(f" Encounter methods: {len(mapped_methods)} mapped, {len(unmapped_methods)} unmapped")
if unmapped_methods:
print(" Unmapped methods:", file=sys.stderr)
for m in sorted(unmapped_methods):
print(f" - {m}", file=sys.stderr)
# Spot-check pokemon mapping on actual encounter data
form_ids_in_encounters: set[str] = set()
for e in pokedb.encounters:
fid = e.get("pokemon_form_identifier")
if fid:
form_ids_in_encounters.add(fid)
mapped_forms = 0
for fid in form_ids_in_encounters:
if pokemon_mapper.lookup(fid) is not None:
mapped_forms += 1
total_forms = len(form_ids_in_encounters)
print(f" Pokemon forms: {mapped_forms}/{total_forms} mapped from encounters")
pokemon_mapper.report_unmapped()
# Process encounters per game
print("\nProcessing encounters...")
games_to_process: list[tuple[str, str, int]] = [] # (vg_key, game_slug, generation)
for vg_key, vg_info in config.version_groups.items():
generation = vg_info.get("generation", 0)
for slug in vg_info.get("versions", []):
if target_game and slug != target_game:
continue
games_to_process.append((vg_key, slug, generation))
all_encountered_form_ids: set[str] = set()
for vg_key, game_slug, generation in games_to_process:
print(f"\n--- {game_slug} ---")
# Filter encounters for this game
game_encounters = filter_encounters_for_game(pokedb.encounters, game_slug)
if not game_encounters:
print(f" No encounters found in PokeDB data")
continue
print(f" Raw encounters: {len(game_encounters)}")
# Track encountered form IDs for pokemon.json
for e in game_encounters:
fid = e.get("pokemon_form_identifier")
if fid:
all_encountered_form_ids.add(fid)
# Process into grouped encounters
encounters_by_area = process_encounters(
game_encounters, generation, pokemon_mapper, location_mapper,
)
print(f" Location areas with encounters: {len(encounters_by_area)}")
# Build route hierarchy
routes = build_routes(encounters_by_area, location_mapper)
# Merge special encounters (starters, gifts, fossils)
routes = merge_special_encounters(routes, config, vg_key, pokemon_mapper)
# Sort routes by game progression order
routes = sort_routes(routes, config, vg_key)
# Stats
total_routes = sum(1 + len(r.children) for r in routes)
total_enc = sum(
len(r.encounters) + sum(len(c.encounters) for c in r.children)
for r in routes
)
print(f" Routes: {total_routes}")
print(f" Encounter entries: {total_enc}")
# Write per-game JSON
write_game_json(routes, output_dir, game_slug)
# Download sprites to frontend/public/sprites
sprites_dir = find_sprites_dir()
print(f"\nDownloading sprites to {sprites_dir}...")
sprite_map = download_sprites(pokemon_mapper, all_encountered_form_ids, sprites_dir)
print(f" Sprite map covers {len(sprite_map)} forms")
# Download sprites for ALL pokemon (including non-encountered evolutions etc.)
all_sprite_ids = download_all_sprites(pokemon_mapper, sprites_dir)
# Write global JSON files
print("\nWriting global data files...")
write_games_json(config, output_dir)
write_pokemon_json(pokemon_mapper, all_encountered_form_ids, sprite_map, output_dir)
print("\nDone!")
if __name__ == "__main__":
main()
+168
View File
@@ -0,0 +1,168 @@
"""Load and validate PokeDB JSON export files."""
from __future__ import annotations
import json
import sys
import urllib.request
from pathlib import Path
from typing import Any
_CDN_BASE = "https://cdn.pokedb.org"
REQUIRED_FILES: dict[str, str] = {
"encounters.json": f"{_CDN_BASE}/data_export_encounters_json",
"locations.json": f"{_CDN_BASE}/data_export_locations_json",
"location_areas.json": f"{_CDN_BASE}/data_export_location_areas_json",
"encounter_methods.json": f"{_CDN_BASE}/data_export_encounter_methods_json",
"versions.json": f"{_CDN_BASE}/data_export_versions_json",
"pokemon_forms.json": f"{_CDN_BASE}/data_export_pokemon_forms_json",
}
class PokeDBData:
"""Container for all loaded PokeDB export data."""
def __init__(
self,
encounters: list[dict[str, Any]],
locations: list[dict[str, Any]],
location_areas: list[dict[str, Any]],
encounter_methods: list[dict[str, Any]],
versions: list[dict[str, Any]],
pokemon_forms: list[dict[str, Any]],
) -> None:
self.encounters = encounters
self.locations = locations
self.location_areas = location_areas
self.encounter_methods = encounter_methods
self.versions = versions
self.pokemon_forms = pokemon_forms
def summary(self) -> str:
return (
f"PokeDB data loaded:\n"
f" encounters: {len(self.encounters):,}\n"
f" locations: {len(self.locations):,}\n"
f" location_areas: {len(self.location_areas):,}\n"
f" encounter_methods: {len(self.encounter_methods):,}\n"
f" versions: {len(self.versions):,}\n"
f" pokemon_forms: {len(self.pokemon_forms):,}"
)
def download_pokedb_data(data_dir: Path) -> None:
"""Download missing PokeDB JSON export files into data_dir."""
data_dir.mkdir(parents=True, exist_ok=True)
missing = {f: url for f, url in REQUIRED_FILES.items() if not (data_dir / f).exists()}
if not missing:
return
print(f"Downloading {len(missing)} PokeDB file(s) to {data_dir}...")
for filename, url in missing.items():
dest = data_dir / filename
print(f" {filename}...", end="", flush=True)
try:
urllib.request.urlretrieve(url, dest)
size_mb = dest.stat().st_size / (1024 * 1024)
print(f" {size_mb:.1f} MB")
except Exception as e:
print(f" FAILED", file=sys.stderr)
print(f"Error downloading {url}: {e}", file=sys.stderr)
sys.exit(1)
print()
def load_pokedb_data(data_dir: Path) -> PokeDBData:
"""Load all PokeDB JSON export files from a directory.
Downloads any missing files automatically, then validates and loads them.
"""
download_pokedb_data(data_dir)
missing = [f for f in REQUIRED_FILES if not (data_dir / f).exists()]
if missing:
print(f"Error: Still missing files after download: {missing}", file=sys.stderr)
sys.exit(1)
def _load(filename: str) -> list[dict[str, Any]]:
path = data_dir / filename
try:
with open(path) as f:
data = json.load(f)
except json.JSONDecodeError as e:
print(f"Error: Failed to parse {path}: {e}", file=sys.stderr)
sys.exit(1)
if not isinstance(data, list):
print(
f"Error: Expected a JSON array in {path}, got {type(data).__name__}",
file=sys.stderr,
)
sys.exit(1)
return data
return PokeDBData(
encounters=_load("encounters.json"),
locations=_load("locations.json"),
location_areas=_load("location_areas.json"),
encounter_methods=_load("encounter_methods.json"),
versions=_load("versions.json"),
pokemon_forms=_load("pokemon_forms.json"),
)
class SeedConfig:
"""Container for existing seed configuration files."""
def __init__(
self,
version_groups: dict[str, Any],
route_order: dict[str, list[str]],
special_encounters: dict[str, Any] | None,
) -> None:
self.version_groups = version_groups
self.route_order = route_order
self.special_encounters = special_encounters
def load_seed_config(seeds_dir: Path) -> SeedConfig:
"""Load existing seed configuration files (version_groups, route_order, etc.).
Exits with an error message if required config files are missing.
"""
vg_path = seeds_dir / "version_groups.json"
if not vg_path.exists():
print(f"Error: version_groups.json not found at {vg_path}", file=sys.stderr)
sys.exit(1)
with open(vg_path) as f:
version_groups = json.load(f)
# Load route_order.json and resolve aliases
ro_path = seeds_dir / "route_order.json"
if not ro_path.exists():
print(f"Error: route_order.json not found at {ro_path}", file=sys.stderr)
sys.exit(1)
with open(ro_path) as f:
ro_raw = json.load(f)
route_order: dict[str, list[str]] = dict(ro_raw.get("routes", {}))
for alias, target in ro_raw.get("aliases", {}).items():
if target in route_order:
route_order[alias] = route_order[target]
# Load special_encounters.json (optional)
se_path = seeds_dir / "special_encounters.json"
special_encounters = None
if se_path.exists():
with open(se_path) as f:
special_encounters = json.load(f)
return SeedConfig(
version_groups=version_groups,
route_order=route_order,
special_encounters=special_encounters,
)
@@ -0,0 +1,683 @@
"""Reference data mappings: PokeDB identifiers → seed format values."""
from __future__ import annotations
import json
import re
import sys
from pathlib import Path
from typing import Any
from .loader import PokeDBData
# ---------------------------------------------------------------------------
# Encounter method mapping
# ---------------------------------------------------------------------------
# PokeDB encounter_method_identifier → our simplified method name.
# Keys can be exact matches or prefix patterns (ending with *).
ENCOUNTER_METHOD_MAP: dict[str, str] = {
# Walking / grass / cave
"walking-tall-grass": "walk",
"walking-long-grass": "walk",
"walking-cave": "walk",
"walking-bridge": "walk",
"walking-building": "walk",
"walking-sand": "walk",
"walking-snow": "walk",
"walking-rough-terrain": "walk",
"walking-marsh": "walk",
"walking-puddle": "walk",
"walking-flower-field": "walk",
"walking-ice": "walk",
"walking-forest": "walk",
"walking-snowfield": "walk",
"dark-grass": "walk",
"shaking-grass": "walk",
"rustling-grass": "walk",
"yellow-flowers": "walk",
"red-flowers": "walk",
"purple-flowers": "walk",
# Surfing
"surfing": "surf",
"surfing-ocean": "surf",
"surfing-puddle": "surf",
"surfing-rapids": "surf",
"surfing-underwater": "surf",
"rippling-water": "surf",
# Fishing
"fishing-old-rod": "old-rod",
"fishing-good-rod": "good-rod",
"fishing-super-rod": "super-rod",
"fishing": "fishing",
"fishing-special": "fishing",
# Rock smash
"rock-smash": "rock-smash",
# Headbutt
"headbutt-low": "headbutt",
"headbutt-normal": "headbutt",
"headbutt-high": "headbutt",
# Gift / special acquisition
"npc-gift": "gift",
"egg": "gift",
"revive": "gift",
"fossil": "gift",
# Trade
"npc-trade": "trade",
# Overworld / symbol encounters (Gen 8+)
"symbol-encounter": "walk",
"wanderer": "walk",
"flying": "walk",
# Static / fixed
"fixed-encounter": "static",
"static-encounter": "static",
"legendary-encounter": "static",
"interactable": "static",
# Special methods
"swarm": "swarm",
"poke-radar": "pokeradar",
"dual-slot-mode": "dual-slot",
"honey-tree": "honey",
"trophy-garden": "walk",
"great-marsh": "walk",
"cave-spot": "walk",
"bubble-spot": "surf",
"sand-spot": "walk",
"horde": "walk",
"sos-encounter": "walk",
"ambush": "walk",
# Seaweed / diving
"diving": "surf",
"diving-seaweed": "surf",
"seaweed": "surf",
# Raids
"max-raid": "raid",
"max-raid-battle": "raid",
"dynamax-adventure": "raid",
"tera-raid": "raid",
"tera-raid-battle": "raid",
"fixed-tera-encounter": "static",
# Misc
"roaming": "roaming",
"safari-zone": "walk",
"bug-contest": "walk",
"dust-cloud": "walk",
"hidden-grotto": "static",
"hidden-encounter": "walk",
"horde-encounter": "walk",
"shaking-trees": "walk",
"shaking-ore-deposits": "walk",
"island-scan": "static",
"mass-outbreak": "swarm",
"npc-buy": "gift",
"special-encounter": "static",
"sea-skim": "surf",
"midair": "walk",
"mr-backlot": "walk",
"hoenn-sound": "walk",
"sinnoh-sound": "walk",
"curry": "gift",
"boxes": "gift",
"berry-tree": "walk",
"zygarde-cube-assemble": "static",
"contact-flock": "walk",
"contact-space-time-distortion": "walk",
"contact-unown-reasearch-notes": "static",
"flying-pokemon-shadow": "walk",
}
# Prefix-based fallbacks for methods not explicitly listed above.
_METHOD_PREFIX_MAP: list[tuple[str, str]] = [
("walking-", "walk"),
("surfing-", "surf"),
("fishing-", "fishing"),
("headbutt-", "headbutt"),
("flying-", "walk"),
("ambush-", "walk"),
("contact-", "walk"),
]
def map_encounter_method(method_identifier: str) -> str | None:
"""Map a PokeDB encounter method to our simplified method name.
Returns None if the method is unrecognized.
"""
if method_identifier in ENCOUNTER_METHOD_MAP:
return ENCOUNTER_METHOD_MAP[method_identifier]
for prefix, mapped in _METHOD_PREFIX_MAP:
if method_identifier.startswith(prefix):
return mapped
return None
# ---------------------------------------------------------------------------
# Version mapping
# ---------------------------------------------------------------------------
# PokeDB version identifiers that differ from our game slugs.
# Most are 1:1, these handle exceptions.
_VERSION_OVERRIDES: dict[str, str] = {
"lets-go-pikachu": "lets-go-pikachu",
"lets-go-eevee": "lets-go-eevee",
}
def build_version_map(
pokedb: PokeDBData,
version_groups: dict[str, Any],
) -> dict[str, str]:
"""Build a mapping from PokeDB version_identifier → our game slug.
Returns the mapping dict. Logs warnings for unmapped versions.
"""
# Collect all our known game slugs
our_slugs: set[str] = set()
for vg in version_groups.values():
for slug in vg.get("versions", []):
our_slugs.add(slug)
# Collect all PokeDB version identifiers
pokedb_versions: set[str] = set()
for v in pokedb.versions:
identifier = v.get("identifier", "")
if identifier:
pokedb_versions.add(identifier)
mapping: dict[str, str] = {}
for pdb_ver in sorted(pokedb_versions):
if pdb_ver in _VERSION_OVERRIDES:
mapping[pdb_ver] = _VERSION_OVERRIDES[pdb_ver]
elif pdb_ver in our_slugs:
mapping[pdb_ver] = pdb_ver
# else: PokeDB version not in our version_groups (expected for some)
# Report our games that have no PokeDB mapping
mapped_slugs = set(mapping.values())
unmapped_ours = our_slugs - mapped_slugs
if unmapped_ours:
print(f" Versions in our config with no PokeDB match: {sorted(unmapped_ours)}")
return mapping
# ---------------------------------------------------------------------------
# Pokemon form mapping
# ---------------------------------------------------------------------------
# PokeDB uses adjectival region forms ("alolan") while PokeAPI/our data uses
# region names ("alola"). This maps PokeDB suffixes → our suffixes.
_FORM_SUFFIX_MAP: dict[str, str] = {
"alolan": "alola",
"galarian": "galar",
"hisuian": "hisui",
"paldean": "paldea",
# Totem forms
"alolan-totem": "totem-alola",
# Basculin stripes
"blue-stripe": "blue-striped",
"red-stripe": "red-striped",
"white-stripe": "white-striped",
# Sea forms
"west-sea": "west",
"east-sea": "east",
# Cloak forms
"plant-cloak": "plant",
"sandy-cloak": "sandy",
"trash-cloak": "trash",
# Eiscue
"ice-face": "ice",
# Misc forms
"pompom": "pom-pom",
"10p": "10",
"50p": "50",
"owntempo": "own-tempo",
"two": "two-segment",
"chest": "chest-form",
"ice-rider": "ice",
"shadow-rider": "shadow",
"apex": "apex-build",
"ultimate": "ultimate-mode",
"black-activated": "black",
"white-activated": "white",
"hero": "hero",
"sword": "crowned",
"shield": "crowned",
# Gigantamax
"gigantamax": "gmax",
# Partner forms
"partner": "partner-cap",
# Flabébé / Floette / Florges color forms — these don't have form suffixes in our data
# since each color is just the base form. Map to base.
"blue": "blue",
"orange": "orange",
"red": "red",
"white": "white",
"yellow": "yellow",
# Gender forms
"female": "female",
"male": "male",
# Furfrou
"natural": "natural",
# Cherrim
"overcast": "overcast",
# Sinistea / Polteageist
"antique": "antique",
"phony": "phony",
# Poltchageist / Sinistcha
"artisan": "artisan",
"counterfeit": "counterfeit",
"masterpiece": "masterpiece",
"unremarkable": "unremarkable",
# Minior cores
"blue-core": "blue",
"green-core": "green",
"indigo-core": "indigo",
"orange-core": "orange",
"red-core": "red",
"violet-core": "violet",
"yellow-core": "yellow",
# Vivillon
"fancy": "fancy",
# Squawkabilly
# these use same name
# Xerneas
"neutral": "neutral",
# Deerling / Sawsbuck
"spring": "spring",
"summer": "summer",
"autumn": "autumn",
"winter": "winter",
# Spiky-ears Pichu
"spiky-ears": "spiky-eared",
# Paldean breeds
"paldean-combat-breed": "paldea-combat-breed",
"paldean-blaze-breed": "paldea-blaze-breed",
"paldean-aqua-breed": "paldea-aqua-breed",
}
# PokeDB type IDs → type names (from PokeDB's type system)
TYPE_ID_MAP: dict[int, str] = {
1: "normal",
2: "fighting",
3: "flying",
4: "poison",
5: "ground",
6: "rock",
7: "bug",
8: "ghost",
9: "steel",
10: "fire",
11: "water",
12: "grass",
13: "electric",
14: "psychic",
15: "ice",
16: "dragon",
17: "dark",
18: "fairy",
}
def _normalize_slug(identifier: str) -> str:
"""Normalize a PokeDB pokemon_form_identifier to a PokeAPI-style slug.
PokeDB uses "pidgey-default" for base forms strip the "-default" suffix.
For alternate forms, translate PokeDB naming conventions to ours.
"""
if identifier.endswith("-default"):
return identifier[: -len("-default")]
# Try suffix-based mapping: split into species + form suffix
# e.g. "rattata-alolan" → species="rattata", suffix="alolan"
# e.g. "mr-mime-galarian" → need to find the right split point
# Strategy: try longest suffix first
for pokedb_suffix, our_suffix in sorted(
_FORM_SUFFIX_MAP.items(), key=lambda x: -len(x[0])
):
if identifier.endswith("-" + pokedb_suffix):
species = identifier[: -(len(pokedb_suffix) + 1)]
return f"{species}-{our_suffix}"
return identifier
def _name_to_slug(name: str) -> str:
"""Convert a display name to a PokeAPI-style slug.
"Bulbasaur" "bulbasaur"
"Mr. Mime" "mr-mime"
"Farfetch'd" "farfetchd"
"Nidoran♀" "nidoran-f"
"Nidoran♂" "nidoran-m"
"Flabébé" "flabebe"
"Type: Null" "type-null"
"""
s = name.lower()
s = s.replace("", "-f").replace("", "-m")
s = s.replace("'", "").replace("'", "").replace(".", "").replace(":", "")
s = s.replace("é", "e").replace("É", "e")
s = s.replace(" ", "-")
# Collapse multiple hyphens
s = re.sub(r"-+", "-", s)
return s.strip("-")
def _name_to_form_slug(name: str) -> str | None:
"""Convert a display name with form suffix to a PokeAPI-style slug.
"Rattata (Alola)" "rattata-alola"
"Basculin (Blue Striped)" "basculin-blue-striped"
"Deoxys Normal" "deoxys-normal" (space-separated variant)
"""
# Try parenthesized form: "Base (Suffix)"
m = re.match(r"^(.+?)\s*\((.+)\)$", name)
if m:
base = _name_to_slug(m.group(1))
suffix = _name_to_slug(m.group(2))
return f"{base}-{suffix}"
# Try space-separated form: "Deoxys Normal"
parts = name.split()
if len(parts) >= 2:
return _name_to_slug(name)
return None
# Manual overrides for PokeDB identifiers that can't be resolved generically.
# These are cases where our pokemon.json uses non-standard base form names
# (e.g. "Deoxys Normal" instead of "Deoxys").
_FORM_OVERRIDES: dict[str, tuple[int, str]] = {
"deoxys-default": (386, "Deoxys Normal"),
"darmanitan-galarian": (10177, "Darmanitan (Galar Standard)"),
"mimikyu-totem": (10144, "Mimikyu (Totem Disguised)"),
"squawkabilly-green": (931, "Squawkabilly Green Plumage"),
"squawkabilly-blue": (10260, "Squawkabilly (Blue Plumage)"),
"squawkabilly-white": (10262, "Squawkabilly (White Plumage)"),
"squawkabilly-yellow": (10261, "Squawkabilly (Yellow Plumage)"),
"toxtricity-gigantamax": (849, "Toxtricity Amped"),
}
class PokemonMapper:
"""Maps PokeDB pokemon_form_identifier → (pokeapi_id, display_name)."""
def __init__(self, pokemon_json_path: Path, pokedb: PokeDBData) -> None:
# Build slug → (pokeapi_id, name) from existing pokemon.json
self._slug_to_info: dict[str, tuple[int, str]] = {}
self._id_to_info: dict[int, tuple[int, str]] = {} # pokeapi_id → (national_dex, name)
self._existing_types: dict[int, list[str]] = {} # pokeapi_id → types (fallback)
self._unmapped: set[str] = set()
if pokemon_json_path.exists():
with open(pokemon_json_path) as f:
pokemon_list = json.load(f)
for p in pokemon_list:
pid = p["pokeapi_id"]
name = p["name"]
ndex = p["national_dex"]
self._id_to_info[pid] = (ndex, name)
if p.get("types"):
self._existing_types[pid] = p["types"]
# Index by base slug (from pokeapi_id for base forms)
slug = _name_to_slug(name)
self._slug_to_info[slug] = (pid, name)
# Also index by form slug if it has a form suffix
form_slug = _name_to_form_slug(name)
if form_slug and form_slug != slug:
self._slug_to_info[form_slug] = (pid, name)
# Build index from PokeDB pokemon_forms.json if it has useful fields
self._pokedb_form_index: dict[str, dict] = {}
# Reverse index: pokeapi_id → PokeDB form record (for non-encountered lookups)
self._id_to_pokedb_form: dict[int, dict] = {}
for form in pokedb.pokemon_forms:
identifier = form.get("identifier", "")
if identifier:
self._pokedb_form_index[identifier] = form
# Build reverse index from pokeapi_id → PokeDB form
# First, for all encountered lookups that succeed, we cache the mapping.
# Here we pre-build for default forms using ndex_id.
for form in pokedb.pokemon_forms:
ndex = form.get("ndex_id")
if ndex and form.get("is_default_form"):
# Default form matches the base species (ndex == pokeapi_id for base forms)
if ndex in self._id_to_info:
self._id_to_pokedb_form[ndex] = form
# Also look for alternate-form pokeapi_ids that share the same ndex
for pid, (p_ndex, _) in self._id_to_info.items():
if p_ndex == ndex and pid not in self._id_to_pokedb_form:
self._id_to_pokedb_form[pid] = form
# Map non-default forms to their specific pokeapi_ids where possible
for form in pokedb.pokemon_forms:
identifier = form.get("identifier", "")
if not identifier or form.get("is_default_form"):
continue
slug = _normalize_slug(identifier)
if slug in self._slug_to_info:
pid, _ = self._slug_to_info[slug]
self._id_to_pokedb_form[pid] = form
def lookup(self, pokemon_form_identifier: str | None) -> tuple[int, str] | None:
"""Look up a PokeDB pokemon_form_identifier.
Returns (pokeapi_id, display_name) or None if unmapped.
"""
if not pokemon_form_identifier:
return None
# Check manual overrides first
if pokemon_form_identifier in _FORM_OVERRIDES:
return _FORM_OVERRIDES[pokemon_form_identifier]
slug = _normalize_slug(pokemon_form_identifier)
# Direct slug match
if slug in self._slug_to_info:
return self._slug_to_info[slug]
# Try the PokeDB form record for a pokemon_id field
form_record = self._pokedb_form_index.get(pokemon_form_identifier, {})
pokemon_id = form_record.get("pokemon_id")
if pokemon_id and pokemon_id in self._id_to_info:
ndex, name = self._id_to_info[pokemon_id]
# Cache for future lookups
self._slug_to_info[slug] = (pokemon_id, name)
return (pokemon_id, name)
# Fallback: strip form suffix to find base species.
# Many cosmetic forms (colors, genders, seasons) don't have separate
# entries in our pokemon.json — they use the base species entry.
# Try progressively shorter slugs: "flabebe-blue" → "flabebe"
parts = slug.split("-")
for i in range(len(parts) - 1, 0, -1):
base = "-".join(parts[:i])
if base in self._slug_to_info:
result = self._slug_to_info[base]
# Cache for future lookups
self._slug_to_info[slug] = result
return result
# Track unmapped
if pokemon_form_identifier not in self._unmapped:
self._unmapped.add(pokemon_form_identifier)
return None
def lookup_by_id(self, pokeapi_id: int) -> tuple[int, str] | None:
"""Look up a pokemon by its pokeapi_id.
Returns (pokeapi_id, display_name) or None if not found.
"""
if pokeapi_id in self._id_to_info:
_, name = self._id_to_info[pokeapi_id]
return (pokeapi_id, name)
return None
def get_sprite_url(self, pokemon_form_identifier: str | None) -> str | None:
"""Get the PokeDB CDN sprite URL (100x100 medium) for a form identifier."""
if not pokemon_form_identifier:
return None
form_record = self._pokedb_form_index.get(pokemon_form_identifier, {})
return form_record.get("main_image_normal_path_medium")
def get_form_data(self, pokemon_form_identifier: str | None) -> dict | None:
"""Get the full PokeDB form record for a form identifier."""
if not pokemon_form_identifier:
return None
return self._pokedb_form_index.get(pokemon_form_identifier)
def all_pokemon(self) -> list[tuple[int, tuple[int, str]]]:
"""Return all known pokemon as [(pokeapi_id, (national_dex, name)), ...].
Sourced from the existing pokemon.json.
"""
return sorted(self._id_to_info.items())
def get_types_for_id(self, pokeapi_id: int) -> list[str]:
"""Get types for a pokemon by pokeapi_id, looking up via PokeDB form data.
Falls back to existing types from pokemon.json if no PokeDB form found.
"""
form = self._find_form_for_id(pokeapi_id)
if form:
types = []
t1 = form.get("type_1_id")
t2 = form.get("type_2_id")
if t1 and t1 in TYPE_ID_MAP:
types.append(TYPE_ID_MAP[t1])
if t2 and t2 in TYPE_ID_MAP:
types.append(TYPE_ID_MAP[t2])
if types:
return types
# Fallback to existing types from pokemon.json
return self._existing_types.get(pokeapi_id, [])
def _find_form_for_id(self, pokeapi_id: int) -> dict | None:
"""Find the PokeDB form record for a pokeapi_id."""
# Check pre-built reverse index first
if pokeapi_id in self._id_to_pokedb_form:
return self._id_to_pokedb_form[pokeapi_id]
if pokeapi_id not in self._id_to_info:
return None
_, name = self._id_to_info[pokeapi_id]
slug = _name_to_slug(name)
for suffix in ["-default", ""]:
candidate = slug + suffix
if candidate in self._pokedb_form_index:
return self._pokedb_form_index[candidate]
form_slug = _name_to_form_slug(name)
if form_slug:
for suffix in ["-default", ""]:
candidate = form_slug + suffix
if candidate in self._pokedb_form_index:
return self._pokedb_form_index[candidate]
return None
def has_sprite_for_id(self, pokeapi_id: int) -> bool:
"""Check if a sprite exists for a pokemon by pokeapi_id."""
form = self._find_form_for_id(pokeapi_id)
return bool(form and form.get("main_image_normal_path_medium"))
def get_sprite_url_for_id(self, pokeapi_id: int) -> str | None:
"""Get the PokeDB CDN sprite URL for a pokemon by pokeapi_id."""
form = self._find_form_for_id(pokeapi_id)
return form.get("main_image_normal_path_medium") if form else None
def report_unmapped(self) -> None:
"""Print warnings for any unmapped identifiers."""
if self._unmapped:
print(
f"\nWarning: {len(self._unmapped)} unmapped pokemon form identifiers:",
file=sys.stderr,
)
for ident in sorted(self._unmapped):
print(f" - {ident}", file=sys.stderr)
# ---------------------------------------------------------------------------
# Location area mapping
# ---------------------------------------------------------------------------
# Region prefixes to strip from location identifiers (matching Go tool behavior).
_REGION_PREFIXES = [
"kanto-", "johto-", "hoenn-", "sinnoh-",
"unova-", "kalos-", "alola-", "galar-",
"hisui-", "paldea-",
]
def _identifier_to_name(identifier: str) -> str:
"""Convert a hyphenated identifier to a Title Case display name.
"route-01-kanto" "Route 01 Kanto" (region stripping done separately)
"viridian-forest" "Viridian Forest"
"""
return identifier.replace("-", " ").title()
class LocationMapper:
"""Maps PokeDB location_area_identifier → (location_name, area_suffix)."""
def __init__(self, pokedb: PokeDBData) -> None:
# Build location_area_identifier → location_identifier lookup
self._area_to_location: dict[str, str] = {}
# location_identifier → location display name
self._location_names: dict[str, str] = {}
# location_area_identifier → area display name
self._area_names: dict[str, str] = {}
# Index locations
for loc in pokedb.locations:
identifier = loc.get("identifier", "")
name = loc.get("name", "")
if identifier:
self._location_names[identifier] = name if name else self._clean_location_name(identifier)
# Index location areas
for area in pokedb.location_areas:
area_id = area.get("identifier", "")
loc_id = area.get("location_identifier", "")
area_name = area.get("name", "")
if area_id:
self._area_to_location[area_id] = loc_id
self._area_names[area_id] = area_name if area_name else ""
@staticmethod
def _clean_location_name(identifier: str) -> str:
"""Clean a location identifier into a display name."""
name = identifier
for prefix in _REGION_PREFIXES:
if name.startswith(prefix):
name = name[len(prefix):]
break
return _identifier_to_name(name)
def get_location_name(self, location_area_identifier: str) -> str:
"""Get the display name for a location area's parent location."""
loc_id = self._area_to_location.get(location_area_identifier, "")
if loc_id and loc_id in self._location_names:
return self._location_names[loc_id]
# Fallback: derive from the area identifier itself
return self._clean_location_name(location_area_identifier)
def get_area_name(self, location_area_identifier: str) -> str:
"""Get the display name for a specific location area."""
return self._area_names.get(location_area_identifier, "")
def get_location_identifier(self, location_area_identifier: str) -> str:
"""Get the parent location identifier for a location area."""
return self._area_to_location.get(location_area_identifier, "")
@@ -0,0 +1,81 @@
"""Output data models matching the existing seed JSON format."""
from __future__ import annotations
from dataclasses import dataclass, field
@dataclass
class Encounter:
pokeapi_id: int
pokemon_name: str
method: str
encounter_rate: int
min_level: int
max_level: int
def to_dict(self) -> dict:
return {
"pokeapi_id": self.pokeapi_id,
"pokemon_name": self.pokemon_name,
"method": self.method,
"encounter_rate": self.encounter_rate,
"min_level": self.min_level,
"max_level": self.max_level,
}
@dataclass
class Route:
name: str
order: int
encounters: list[Encounter] = field(default_factory=list)
children: list[Route] = field(default_factory=list)
def to_dict(self) -> dict:
d: dict = {
"name": self.name,
"order": self.order,
"encounters": [e.to_dict() for e in self.encounters],
}
if self.children:
d["children"] = [c.to_dict() for c in self.children]
return d
@dataclass
class Game:
name: str
slug: str
generation: int
region: str
release_year: int
color: str | None = None
def to_dict(self) -> dict:
return {
"name": self.name,
"slug": self.slug,
"generation": self.generation,
"region": self.region,
"release_year": self.release_year,
"color": self.color,
}
@dataclass
class Pokemon:
pokeapi_id: int
national_dex: int
name: str
types: list[str]
sprite_url: str
def to_dict(self) -> dict:
return {
"pokeapi_id": self.pokeapi_id,
"national_dex": self.national_dex,
"name": self.name,
"types": self.types,
"sprite_url": self.sprite_url,
}
+289
View File
@@ -0,0 +1,289 @@
"""Output seed JSON files in the existing format."""
from __future__ import annotations
import json
import re
import sys
from pathlib import Path
from typing import Any
from .loader import SeedConfig
from .mappings import TYPE_ID_MAP, PokemonMapper
from .models import Encounter, Route
from .sprites import sprite_path_for_pokemon
# Max route name length (matches DB column VARCHAR(100))
_MAX_ROUTE_NAME_LEN = 100
# Pattern for Sw/Sh den area names: "Location (Den X - long description - Common/Rare)"
_DEN_NAME_RE = re.compile(r"^(.+?) \(Den ([A-Z]\d*) - .+ - (Common|Rare)\)$")
def _truncate_route_name(name: str) -> str:
"""Truncate a route name to fit the database column limit.
Applies smart truncation for known patterns (e.g. Sw/Sh den descriptions).
"""
if len(name) <= _MAX_ROUTE_NAME_LEN:
return name
# Sw/Sh dens: shorten "Location (Den X - long desc - Common/Rare)" → "Location (Den X - Common/Rare)"
m = _DEN_NAME_RE.match(name)
if m:
shortened = f"{m.group(1)} (Den {m.group(2)} - {m.group(3)})"
if len(shortened) <= _MAX_ROUTE_NAME_LEN:
return shortened
# Generic truncation: cut at last space before limit, add ellipsis
truncated = name[:_MAX_ROUTE_NAME_LEN - 1].rsplit(" ", 1)[0] + ""
return truncated
# ---------------------------------------------------------------------------
# Route ordering
# ---------------------------------------------------------------------------
def _get_route_order(config: SeedConfig, vg_key: str) -> list[str] | None:
"""Get the route order list for a version group, resolving aliases."""
return config.route_order.get(vg_key)
def _route_sort_key(name: str, order_list: list[str]) -> tuple[int, str]:
"""Compute sort key for a route name against an ordered list."""
for i, ordered_name in enumerate(order_list):
if name == ordered_name or name.startswith(ordered_name + " ("):
return (i, name)
return (len(order_list), name)
def sort_routes(routes: list[Route], config: SeedConfig, vg_key: str) -> list[Route]:
"""Sort routes by game progression order and assign sequential order values."""
order_list = _get_route_order(config, vg_key)
if order_list:
routes.sort(key=lambda r: _route_sort_key(r.name, order_list))
else:
print(f" Warning: No route order for {vg_key}, using default order", file=sys.stderr)
# Assign sequential order values
order = 1
for route in routes:
route.order = order
order += 1
for child in route.children:
child.order = order
order += 1
return routes
# ---------------------------------------------------------------------------
# Special encounters
# ---------------------------------------------------------------------------
def _get_special_encounters(
config: SeedConfig,
vg_key: str,
) -> dict[str, list[dict[str, Any]]] | None:
"""Get special encounters for a version group, resolving aliases."""
if not config.special_encounters:
return None
encounters = config.special_encounters.get("encounters", {})
aliases = config.special_encounters.get("aliases", {})
if vg_key in encounters:
return encounters[vg_key]
if vg_key in aliases:
alias_target = aliases[vg_key]
if alias_target in encounters:
return encounters[alias_target]
return None
def merge_special_encounters(
routes: list[Route],
config: SeedConfig,
vg_key: str,
pokemon_mapper: PokemonMapper | None = None,
) -> list[Route]:
"""Merge special encounters (starters, gifts, fossils) into routes."""
special_data = _get_special_encounters(config, vg_key)
if not special_data:
return routes
# Build lookup: route name → route (including children)
route_map: dict[str, Route] = {}
for route in routes:
route_map[route.name] = route
for child in route.children:
route_map[child.name] = child
for location_name, encounter_dicts in special_data.items():
encounters = []
for e in encounter_dicts:
# Look up proper display name from pokemon mapper if available
pokemon_name = e["pokemon_name"]
if pokemon_mapper:
info = pokemon_mapper.lookup_by_id(e["pokeapi_id"])
if info:
pokemon_name = info[1]
encounters.append(Encounter(
pokeapi_id=e["pokeapi_id"],
pokemon_name=pokemon_name,
method=e["method"],
encounter_rate=e["encounter_rate"],
min_level=e["min_level"],
max_level=e["max_level"],
))
if location_name in route_map:
route_map[location_name].encounters.extend(encounters)
else:
new_route = Route(name=location_name, order=0, encounters=encounters)
routes.append(new_route)
route_map[location_name] = new_route
return routes
# ---------------------------------------------------------------------------
# JSON output
# ---------------------------------------------------------------------------
def write_json(path: Path, data: Any) -> None:
"""Write data as formatted JSON with trailing newline."""
content = json.dumps(data, indent=2, ensure_ascii=False)
path.write_text(content + "\n", encoding="utf-8")
print(f" -> {path}")
def _deduplicate_names(routes: list[Route]) -> None:
"""Ensure all route names are unique by appending a numeric suffix to duplicates."""
seen: dict[str, int] = {}
def _unique(name: str) -> str:
if name not in seen:
seen[name] = 1
return name
seen[name] += 1
return f"{name} #{seen[name]}"
for route in routes:
route.name = _unique(_truncate_route_name(route.name))
for child in route.children:
child.name = _unique(_truncate_route_name(child.name))
def write_game_json(routes: list[Route], output_dir: Path, game_slug: str) -> None:
"""Write a per-game route/encounter JSON file."""
_deduplicate_names(routes)
data = [r.to_dict() for r in routes]
write_json(output_dir / f"{game_slug}.json", data)
_GAME_CATEGORY: dict[str, str] = {
"red": "original", "blue": "original", "yellow": "enhanced",
"gold": "original", "silver": "original", "crystal": "enhanced",
"ruby": "original", "sapphire": "original", "emerald": "enhanced",
"firered": "remake", "leafgreen": "remake",
"diamond": "original", "pearl": "original", "platinum": "enhanced",
"heartgold": "remake", "soulsilver": "remake",
"black": "original", "white": "original",
"black-2": "sequel", "white-2": "sequel",
"x": "original", "y": "original",
"omega-ruby": "remake", "alpha-sapphire": "remake",
"sun": "original", "moon": "original",
"ultra-sun": "enhanced", "ultra-moon": "enhanced",
"lets-go-pikachu": "remake", "lets-go-eevee": "remake",
"sword": "original", "shield": "original",
"brilliant-diamond": "remake", "shining-pearl": "remake",
"legends-arceus": "spinoff",
"scarlet": "original", "violet": "original",
"legends-z-a": "spinoff",
}
def write_games_json(config: SeedConfig, output_dir: Path) -> None:
"""Write games.json from version_groups config."""
games = []
for vg_info in config.version_groups.values():
for game_info in vg_info.get("games", {}).values():
slug = game_info["slug"]
games.append({
"name": game_info["name"],
"slug": slug,
"generation": vg_info["generation"],
"region": vg_info["region"],
"release_year": game_info["release_year"],
"color": game_info.get("color"),
"category": _GAME_CATEGORY.get(slug, "original"),
})
write_json(output_dir / "games.json", games)
print(f" Wrote {len(games)} games")
def write_pokemon_json(
pokemon_mapper: PokemonMapper,
encountered_form_ids: set[str],
sprite_map: dict[str, str],
output_dir: Path,
) -> None:
"""Write pokemon.json with all known pokemon.
Includes all pokemon from the existing pokemon.json (base data),
enriched with PokeDB types and sprite paths for encountered forms.
"""
# Build a mapping of pokeapi_id → (form_id, form_data) for encountered forms
encountered_by_id: dict[int, tuple[str, dict[str, Any] | None]] = {}
for form_id in encountered_form_ids:
info = pokemon_mapper.lookup(form_id)
if info is None:
continue
pokeapi_id, _ = info
if pokeapi_id not in encountered_by_id:
form_data = pokemon_mapper.get_form_data(form_id)
encountered_by_id[pokeapi_id] = (form_id, form_data)
pokemon_list: list[dict[str, Any]] = []
for pokeapi_id, (ndex, name) in pokemon_mapper.all_pokemon():
# Enrich with PokeDB data if this pokemon was encountered
if pokeapi_id in encountered_by_id:
form_id, form_data = encountered_by_id[pokeapi_id]
types = _extract_types(form_data) if form_data else []
sprite_url = sprite_path_for_pokemon(pokeapi_id) if form_id in sprite_map else None
national_dex = form_data.get("ndex_id", ndex) if form_data else ndex
else:
# Not encountered — use existing data, try to find PokeDB form for types
types = pokemon_mapper.get_types_for_id(pokeapi_id)
sprite_url = sprite_path_for_pokemon(pokeapi_id) if pokemon_mapper.has_sprite_for_id(pokeapi_id) else None
national_dex = ndex
pokemon_list.append({
"pokeapi_id": pokeapi_id,
"national_dex": national_dex,
"name": name,
"types": types,
"sprite_url": sprite_url,
})
# Sort by pokeapi_id
pokemon_list.sort(key=lambda p: p["pokeapi_id"])
write_json(output_dir / "pokemon.json", pokemon_list)
print(f" Wrote {len(pokemon_list)} pokemon")
def _extract_types(form_data: dict[str, Any]) -> list[str]:
"""Extract type names from a PokeDB form record."""
types = []
type1_id = form_data.get("type_1_id")
type2_id = form_data.get("type_2_id")
if type1_id and type1_id in TYPE_ID_MAP:
types.append(TYPE_ID_MAP[type1_id])
if type2_id and type2_id in TYPE_ID_MAP:
types.append(TYPE_ID_MAP[type2_id])
return types
@@ -0,0 +1,343 @@
"""Core encounter processing: filter, parse, aggregate, and group encounters."""
from __future__ import annotations
import re
from typing import Any
from .mappings import LocationMapper, PokemonMapper, map_encounter_method
from .models import Encounter, Route
# ---------------------------------------------------------------------------
# Rate parsing
# ---------------------------------------------------------------------------
# Word-based rates → numeric value
_WORD_RATES: dict[str, int] = {
"one": 100,
"two": 50,
"three": 33,
"four": 25,
"five": 20,
"six": 17,
"seven": 14,
"eight": 13,
"choose one": 100,
"one of three": 33,
"only one": 100,
"unlimited": 100,
"respawns": 100,
"common": 60,
"average": 30,
"rare": 10,
"varies": 50,
}
_PERCENT_RE = re.compile(r"~?(\d+(?:\.\d+)?)%?")
def parse_rate(value: str | None) -> int | None:
"""Parse a rate string into an integer percentage (0-100).
Handles formats: "50%", "~10%", "one", "common", "100", "??%", etc.
Returns None if unparseable.
"""
if not value:
return None
value = value.strip()
# Word-based
lower = value.lower()
if lower in _WORD_RATES:
return _WORD_RATES[lower]
# Unknown
if value == "??%":
return None
# Numeric percentage: "50%", "~10%", "10.14%", or bare "100"
m = _PERCENT_RE.match(value)
if m:
return max(1, round(float(m.group(1))))
return None
def extract_encounter_rate(record: dict[str, Any], generation: int) -> int:
"""Extract a single encounter_rate from a PokeDB encounter record.
Flattens generation-specific rate variants into a single value.
"""
# Gen 1/3/6: rate_overall
rate_overall = parse_rate(record.get("rate_overall"))
if rate_overall is not None:
return rate_overall
# Gen 2/4: time-of-day rates — take the max
time_rates = [
parse_rate(record.get("rate_morning")),
parse_rate(record.get("rate_day")),
parse_rate(record.get("rate_night")),
]
time_rates = [r for r in time_rates if r is not None]
if time_rates:
return max(time_rates)
# Gen 5: seasonal rates — take the max
season_rates = [
parse_rate(record.get("rate_spring")),
parse_rate(record.get("rate_summer")),
parse_rate(record.get("rate_autumn")),
parse_rate(record.get("rate_winter")),
]
season_rates = [r for r in season_rates if r is not None]
if season_rates:
return max(season_rates)
# Gen 8 Sw/Sh: weather rates — take the max
weather_rates = []
for key, val in record.items():
if key.startswith("weather_") and key.endswith("_rate") and val:
parsed = parse_rate(val)
if parsed is not None:
weather_rates.append(parsed)
if weather_rates:
return max(weather_rates)
# Gen 8 Legends Arceus: boolean conditions → presence-based
if record.get("during_any_time") or record.get("during_morning") or \
record.get("during_day") or record.get("during_evening") or record.get("during_night"):
return 100 # Present under conditions
# Gen 9 Sc/Vi: probability weights → normalize
prob_overall = record.get("probability_overall")
if prob_overall:
parsed = parse_rate(prob_overall)
if parsed is not None:
# These are spawn weights (e.g. "20", "300"), not percentages.
# We'll normalize them later during aggregation when we have
# all encounters for a location. For now, store the raw weight.
return parsed
# Check time-based probability variants
prob_rates = [
parse_rate(record.get("probability_morning")),
parse_rate(record.get("probability_day")),
parse_rate(record.get("probability_evening")),
parse_rate(record.get("probability_night")),
]
prob_rates = [r for r in prob_rates if r is not None]
if prob_rates:
return max(prob_rates)
# Fallback: gift/trade/static encounters with no rate
return 100
# ---------------------------------------------------------------------------
# Level parsing
# ---------------------------------------------------------------------------
def parse_levels(levels_str: str | None) -> tuple[int, int]:
"""Parse a level string into (min_level, max_level).
"2 - 4" (2, 4)
"67" (67, 67)
"44 - 51" (44, 51)
Returns (1, 1) if unparseable.
"""
if not levels_str:
return (1, 1)
levels_str = levels_str.strip()
# Range: "2 - 4" or "2-4"
m = re.match(r"(\d+)\s*-\s*(\d+)", levels_str)
if m:
return (int(m.group(1)), int(m.group(2)))
# Single: "67"
m = re.match(r"(\d+)", levels_str)
if m:
level = int(m.group(1))
return (level, level)
return (1, 1)
# ---------------------------------------------------------------------------
# Core processing
# ---------------------------------------------------------------------------
def filter_encounters_for_game(
encounters: list[dict[str, Any]],
game_slug: str,
) -> list[dict[str, Any]]:
"""Filter PokeDB encounters to only those for a specific game version."""
return [
e for e in encounters
if game_slug in (e.get("version_identifiers") or [])
]
def process_encounters(
raw_encounters: list[dict[str, Any]],
generation: int,
pokemon_mapper: PokemonMapper,
location_mapper: LocationMapper,
) -> dict[str, list[Encounter]]:
"""Process raw PokeDB encounters into grouped-by-location-area Encounter objects.
Returns {location_area_identifier: [Encounter, ...]}.
"""
by_area: dict[str, list[Encounter]] = {}
for record in raw_encounters:
# Map encounter method
method_id = record.get("encounter_method_identifier", "")
method = map_encounter_method(method_id) if method_id else None
if method is None:
continue
# Map pokemon
form_id = record.get("pokemon_form_identifier")
pokemon_info = pokemon_mapper.lookup(form_id)
if pokemon_info is None:
continue
pokeapi_id, pokemon_name = pokemon_info
# Parse levels
min_level, max_level = parse_levels(record.get("levels"))
# Extract rate
encounter_rate = extract_encounter_rate(record, generation)
# Location area
area_id = record.get("location_area_identifier", "")
if not area_id:
continue
enc = Encounter(
pokeapi_id=pokeapi_id,
pokemon_name=pokemon_name,
method=method,
encounter_rate=encounter_rate,
min_level=min_level,
max_level=max_level,
)
by_area.setdefault(area_id, []).append(enc)
return by_area
def aggregate_encounters(encounters: list[Encounter]) -> list[Encounter]:
"""Aggregate encounters by (pokeapi_id, method), merging level ranges and summing rates.
Replicates the Go tool's aggregation logic.
"""
key_type = tuple[int, str]
agg: dict[key_type, Encounter] = {}
order: list[key_type] = []
for enc in encounters:
k = (enc.pokeapi_id, enc.method)
if k in agg:
existing = agg[k]
existing.encounter_rate += enc.encounter_rate
existing.min_level = min(existing.min_level, enc.min_level)
existing.max_level = max(existing.max_level, enc.max_level)
else:
# Copy so we don't mutate the original
agg[k] = Encounter(
pokeapi_id=enc.pokeapi_id,
pokemon_name=enc.pokemon_name,
method=enc.method,
encounter_rate=enc.encounter_rate,
min_level=enc.min_level,
max_level=enc.max_level,
)
order.append(k)
result = []
for k in order:
e = agg[k]
e.encounter_rate = min(e.encounter_rate, 100)
result.append(e)
# Sort by rate descending, then name ascending
result.sort(key=lambda e: (-e.encounter_rate, e.pokemon_name))
return result
def build_routes(
encounters_by_area: dict[str, list[Encounter]],
location_mapper: LocationMapper,
) -> list[Route]:
"""Group encounters by location, building parent/child route hierarchy.
Multiple areas under the same location parent route with children.
Single area flat route.
"""
# Group areas by their parent location identifier
loc_groups: dict[str, list[tuple[str, str, list[Encounter]]]] = {}
# loc_id → [(area_id, area_display_name, encounters), ...]
for area_id, encounters in encounters_by_area.items():
loc_id = location_mapper.get_location_identifier(area_id)
if not loc_id:
loc_id = area_id # fallback
area_name = location_mapper.get_area_name(area_id)
loc_groups.setdefault(loc_id, []).append((area_id, area_name, encounters))
routes: list[Route] = []
for loc_id, areas in loc_groups.items():
loc_name = location_mapper.get_location_name(areas[0][0])
if len(areas) == 1:
# Single area — flat route
_, area_name, encounters = areas[0]
aggregated = aggregate_encounters(encounters)
if aggregated:
# If the area has a distinct name different from the location, use it
route_name = area_name if area_name and area_name != loc_name else loc_name
routes.append(Route(name=route_name, order=0, encounters=aggregated))
else:
# Multiple areas — check if encounters differ
children: list[Route] = []
# Encounters for areas with no distinct name get merged into parent
parent_encounters: list[Encounter] = []
for _, area_name, encounters in areas:
aggregated = aggregate_encounters(encounters)
if aggregated:
if area_name and area_name != loc_name:
children.append(Route(name=area_name, order=0, encounters=aggregated))
else:
# No distinct area name — merge into parent
parent_encounters.extend(aggregated)
if children:
# Parent with children (parent may also have its own encounters)
parent_agg = aggregate_encounters(parent_encounters) if parent_encounters else []
routes.append(Route(
name=loc_name,
order=0,
encounters=parent_agg,
children=children,
))
elif parent_encounters:
# All areas had same name — flatten into single route
routes.append(Route(
name=loc_name,
order=0,
encounters=aggregate_encounters(parent_encounters),
))
return routes
@@ -0,0 +1,130 @@
"""Download and manage PokeDB pokemon sprites."""
from __future__ import annotations
import sys
import urllib.request
from pathlib import Path
from typing import Any
from .mappings import PokemonMapper
def download_sprites(
pokemon_mapper: PokemonMapper,
encountered_form_ids: set[str],
sprites_dir: Path,
) -> dict[str, str]:
"""Download sprites for all encountered pokemon forms.
Returns a mapping of pokemon_form_identifier local sprite filename.
Skips already-downloaded sprites.
"""
sprites_dir.mkdir(parents=True, exist_ok=True)
to_download: list[tuple[str, str, Path]] = [] # (form_id, url, dest)
result: dict[str, str] = {}
for form_id in sorted(encountered_form_ids):
info = pokemon_mapper.lookup(form_id)
if info is None:
continue
pokeapi_id, _ = info
sprite_url = pokemon_mapper.get_sprite_url(form_id)
if not sprite_url:
continue
filename = f"{pokeapi_id}.webp"
dest = sprites_dir / filename
result[form_id] = filename
if not dest.exists():
to_download.append((form_id, sprite_url, dest))
if not to_download:
print(f" Sprites: {len(result)} already cached")
return result
print(f" Downloading {len(to_download)} sprites ({len(result) - len(to_download)} cached)...")
failed = 0
for i, (form_id, url, dest) in enumerate(to_download, 1):
try:
urllib.request.urlretrieve(url, dest)
except Exception as e:
print(f" Warning: Failed to download sprite for {form_id}: {e}", file=sys.stderr)
failed += 1
# Remove the failed entry from results
result.pop(form_id, None)
# Progress every 100
if i % 100 == 0:
print(f" {i}/{len(to_download)}...")
if failed:
print(f" Sprites: {len(result)} downloaded, {failed} failed")
else:
print(f" Sprites: {len(result)} total ({len(to_download)} new)")
return result
def download_all_sprites(
pokemon_mapper: PokemonMapper,
sprites_dir: Path,
) -> set[int]:
"""Download sprites for all known pokemon (not just encountered ones).
Returns a set of pokeapi_ids that have sprites downloaded.
"""
sprites_dir.mkdir(parents=True, exist_ok=True)
to_download: list[tuple[int, str, Path]] = []
have_sprites: set[int] = set()
for pokeapi_id, (_ndex, _name) in pokemon_mapper.all_pokemon():
if pokemon_mapper.has_sprite_for_id(pokeapi_id):
filename = f"{pokeapi_id}.webp"
dest = sprites_dir / filename
have_sprites.add(pokeapi_id)
if not dest.exists():
# Get the sprite URL via the mapper
url = pokemon_mapper.get_sprite_url_for_id(pokeapi_id)
if url:
to_download.append((pokeapi_id, url, dest))
if not to_download:
print(f" All sprites: {len(have_sprites)} already cached")
return have_sprites
print(f" Downloading {len(to_download)} additional sprites ({len(have_sprites) - len(to_download)} cached)...")
failed = 0
for i, (pid, url, dest) in enumerate(to_download, 1):
try:
urllib.request.urlretrieve(url, dest)
except Exception as e:
print(f" Warning: Failed to download sprite for pokemon {pid}: {e}", file=sys.stderr)
failed += 1
have_sprites.discard(pid)
if i % 100 == 0:
print(f" {i}/{len(to_download)}...")
if failed:
print(f" All sprites: {len(have_sprites)} downloaded, {failed} failed")
else:
print(f" All sprites: {len(have_sprites)} total ({len(to_download)} new)")
return have_sprites
def sprite_path_for_pokemon(pokeapi_id: int) -> str:
"""Generate the sprite URL path for use in pokemon.json.
Returns an absolute path like "/sprites/25.webp" for the frontend
(files in frontend/public/ are served at the root).
"""
return f"/sprites/{pokeapi_id}.webp"
+9
View File
@@ -0,0 +1,9 @@
[project]
name = "import-pokedb"
version = "0.1.0"
description = "Convert PokeDB.org JSON data exports into nuzlocke-tracker seed format"
requires-python = ">=3.12"
dependencies = []
[project.scripts]
import-pokedb = "import_pokedb.__main__:main"