Compare commits
8 Commits
| Author | SHA1 | Date | |
|---|---|---|---|
| 90399f41b7 | |||
| aadd730002 | |||
| 872d7872ce | |||
| df55233c62 | |||
| 29b954726a | |||
| d80c59047c | |||
| df7ea64b9e | |||
| 1aa67665ff |
@@ -1,17 +1,22 @@
|
||||
---
|
||||
# nuzlocke-tracker-bs05
|
||||
title: Build PokeDB.org data import tool
|
||||
status: draft
|
||||
type: task
|
||||
status: completed
|
||||
type: feature
|
||||
priority: normal
|
||||
created_at: 2026-02-10T14:04:11Z
|
||||
updated_at: 2026-02-10T14:31:08Z
|
||||
updated_at: 2026-02-11T10:54:04Z
|
||||
parent: nuzlocke-tracker-rzu4
|
||||
blocking:
|
||||
- nuzlocke-tracker-spx3
|
||||
---
|
||||
|
||||
Build a Go tool that converts PokeDB.org's JSON data export into our existing seed JSON format. This replaces PokeAPI as the single source of truth for ALL games (Gen 1-9).
|
||||
Build a standalone Python tool that converts PokeDB.org's JSON data export into our existing seed JSON format. This replaces PokeAPI as the single source of truth for ALL games (Gen 1-9).
|
||||
|
||||
Python was chosen over Go because:
|
||||
- The backend is already Python, so the team is familiar with it
|
||||
- We're processing local JSON files — no need for Go's concurrency
|
||||
- Remains a standalone tool in `tools/import-pokedb/`, not part of the backend
|
||||
|
||||
## Data source
|
||||
|
||||
@@ -64,26 +69,15 @@ Each encounter record has:
|
||||
- `visible` — overworld vs hidden encounter
|
||||
- Max Raid and Tera Raid fields for special encounters
|
||||
|
||||
## Implementation approach
|
||||
## Subtasks
|
||||
|
||||
### Checklist
|
||||
- [ ] Set up project structure in `tools/import-pokedb/`
|
||||
- [ ] Download and cache PokeDB JSON export files
|
||||
- [ ] Parse PokeDB encounters, locations, location_areas, versions, pokemon_forms
|
||||
- [ ] Build lookup maps: pokemon_form_identifier → pokeapi_id (using existing `pokemon.json`)
|
||||
- [ ] Build lookup maps: location_area_identifier → location name + region
|
||||
- [ ] Filter encounters by target game version
|
||||
- [ ] Map PokeDB encounter methods to our seed format methods (73 → simplified set)
|
||||
- [ ] Parse level strings ("2 - 4" → min_level: 2, max_level: 4)
|
||||
- [ ] Handle rate variants per game generation:
|
||||
- For now, flatten time/weather/season rates into `encounter_rate` (use the max or average)
|
||||
- Preserve raw variant data for future use (see nuzlocke-tracker-oqfo)
|
||||
- [ ] Group encounters by location area → route output
|
||||
- [ ] Apply route ordering (use existing route_order.json or generate from location data)
|
||||
- [ ] Output in existing `{game}.json` seed format
|
||||
- [ ] Generate seed data for ALL games, replacing PokeAPI as the single source of truth
|
||||
- [ ] Compare output against existing PokeAPI-sourced data to validate accuracy
|
||||
- [ ] Run for all games and verify output
|
||||
Work is broken into child task beans:
|
||||
|
||||
- [ ] **Set up Python tool scaffold** — project structure, CLI entry point, PokeDB JSON file loading
|
||||
- [ ] **Build reference data mappings** — pokemon_form → pokeapi_id, location_area → name/region, encounter method mapping
|
||||
- [ ] **Core encounter processing** — filter by game version, parse levels, handle rate variants, group by location area
|
||||
- [ ] **Output seed JSON** — produce per-game JSON in existing format, integrate route ordering + special encounters
|
||||
- [ ] **Validation & full generation** — compare against existing data, run for all games, fix discrepancies
|
||||
|
||||
## Encounter method mapping (draft)
|
||||
|
||||
|
||||
@@ -0,0 +1,30 @@
|
||||
---
|
||||
# nuzlocke-tracker-dqyb
|
||||
title: Set up Python tool scaffold
|
||||
status: completed
|
||||
type: task
|
||||
priority: normal
|
||||
created_at: 2026-02-11T08:42:58Z
|
||||
updated_at: 2026-02-11T08:49:55Z
|
||||
parent: nuzlocke-tracker-bs05
|
||||
blocking:
|
||||
- nuzlocke-tracker-zno2
|
||||
---
|
||||
|
||||
Set up the standalone Python tool project in `tools/import-pokedb/`.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [x] Create `tools/import-pokedb/` directory structure
|
||||
- [x] Set up `pyproject.toml` with dependencies (just stdlib should suffice for JSON processing, maybe `click` for CLI)
|
||||
- [x] Create CLI entry point (`__main__.py` or similar) that accepts:
|
||||
- Path to directory containing PokeDB JSON export files
|
||||
- Target output directory (default: `backend/src/app/seeds/data/`)
|
||||
- Optional: specific game version to generate (default: all)
|
||||
- [x] Load and parse all PokeDB JSON files: `encounters.json`, `locations.json`, `location_areas.json`, `encounter_methods.json`, `versions.json`, `pokemon_forms.json`
|
||||
- [x] Basic validation that all expected files are present and parseable
|
||||
|
||||
## Notes
|
||||
- Keep it as a standalone tool, not part of the backend
|
||||
- The PokeDB JSON files are downloaded manually from https://pokedb.org/data-export — no need to automate the download
|
||||
- Model the CLI similarly to how `tools/fetch-pokeapi/` works (cd into dir, run the tool)
|
||||
@@ -0,0 +1,31 @@
|
||||
---
|
||||
# nuzlocke-tracker-gkcy
|
||||
title: Output seed JSON
|
||||
status: completed
|
||||
type: task
|
||||
priority: normal
|
||||
created_at: 2026-02-11T08:43:21Z
|
||||
updated_at: 2026-02-11T10:00:00Z
|
||||
parent: nuzlocke-tracker-bs05
|
||||
blocking:
|
||||
- nuzlocke-tracker-vdks
|
||||
---
|
||||
|
||||
Generate the final per-game JSON files in the existing seed format.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [x] **Apply route ordering**: Use the existing `backend/src/app/seeds/route_order.json` to assign `order` values to routes. Handle aliases (e.g. "red-blue" → "firered-leafgreen"). Log warnings for routes not in the order file.
|
||||
- [x] **Merge special encounters**: Integrate starters, gifts, fossils, and trades from `backend/src/app/seeds/special_encounters.json` into the appropriate routes. Pokemon names are resolved to proper display names via PokemonMapper.
|
||||
- [x] **Output per-game JSON**: Write `{game-slug}.json` files matching the existing format:
|
||||
```json
|
||||
[{"name": "Route 1", "order": 3, "encounters": [...], "children": []}]
|
||||
```
|
||||
- [x] **Output games.json**: Generate the global games list from `version_groups.json` — 38 games written, matching existing count.
|
||||
- [x] **Output pokemon.json**: Generate the global pokemon list including all pokemon referenced in any encounter. Include pokeapi_id, national_dex, name, types, sprite_url.
|
||||
- [x] **Handle version exclusives**: Encounters are filtered by `version_identifiers` per game — verified FireRed vs LeafGreen have 18 exclusives each.
|
||||
|
||||
## Notes
|
||||
- The output must be a drop-in replacement for the existing files in `backend/src/app/seeds/data/`
|
||||
- Boss data (`{game}-bosses.json`) is NOT generated by this tool — it's manually curated
|
||||
- Evolutions data is also separate (currently from PokeAPI) — out of scope for this task
|
||||
@@ -0,0 +1,34 @@
|
||||
---
|
||||
# nuzlocke-tracker-rfg0
|
||||
title: Core encounter processing
|
||||
status: completed
|
||||
type: task
|
||||
priority: normal
|
||||
created_at: 2026-02-11T08:43:12Z
|
||||
updated_at: 2026-02-11T09:12:59Z
|
||||
parent: nuzlocke-tracker-bs05
|
||||
blocking:
|
||||
- nuzlocke-tracker-gkcy
|
||||
---
|
||||
|
||||
Implement the core logic that transforms raw PokeDB encounter records into our internal format.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [x] **Filter by game version**: Given a target game slug, select only encounters where `version_identifiers` includes that game
|
||||
- [x] **Parse level strings**: Convert "2 - 4" → min_level=2, max_level=4; "67" → min_level=67, max_level=67
|
||||
- [x] **Handle rate variants per generation**:
|
||||
- Gen 1/3/6: use `rate_overall` directly as `encounter_rate`
|
||||
- Gen 2/4: `rate_morning`, `rate_day`, `rate_night` — flatten to max or average for `encounter_rate`
|
||||
- Gen 5: `rate_spring` through `rate_winter` — flatten similarly
|
||||
- Gen 8 Sw/Sh: `weather_*_rate` fields — flatten to max
|
||||
- Gen 8 Legends Arceus: `during_*` / `while_*` booleans — convert to a presence-based rate
|
||||
- Gen 9 Sc/Vi: `probability_*` fields (spawn weights, not percentages) — normalize to percentages
|
||||
- Preserve raw variant data in a way that nuzlocke-tracker-oqfo can use later
|
||||
- [x] **Aggregate encounters**: Group by (pokemon, method, location_area) and merge level ranges / rates where appropriate (same logic as the Go tool's aggregation)
|
||||
- [x] **Group by location area**: Collect all encounters for a location area into a route structure
|
||||
- [x] **Handle parent/child routes**: Multi-area locations (e.g. Safari Zone) should produce parent routes with children, matching the existing hierarchical format
|
||||
|
||||
## Notes
|
||||
- Rate parsing needs to handle percentage strings like "40%" as well as bare numbers
|
||||
- The Go tool aggregates encounters with the same pokemon+method at a location into a single entry with merged level ranges — replicate this
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
# nuzlocke-tracker-vdks
|
||||
title: Validation and full generation
|
||||
status: completed
|
||||
type: task
|
||||
priority: normal
|
||||
created_at: 2026-02-11T08:43:29Z
|
||||
updated_at: 2026-02-11T10:52:55Z
|
||||
parent: nuzlocke-tracker-bs05
|
||||
---
|
||||
|
||||
Validate the new tool's output against existing data and generate seed data for all games.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [x] **Diff against existing data**: Compared all 37 games. Route names differ systematically (PokeDB uses proper punctuation/region qualifiers). Encounter counts are generally higher (PokeDB more complete). Version exclusives verified.
|
||||
- [x] **Fix discrepancies**: Fixed: games.json missing `category` field, pokemon.json missing 231 non-encountered pokemon (evolutions etc.), pokemon name casing in special encounters, sprite coverage for all 1350 pokemon, type coverage using fallback to existing data.
|
||||
- [x] **Generate for all games**: All 37 games (+ legends-z-a with no data) generated successfully. All 39 JSON files validated. 7 previously empty games now populated (Sw/Sh, BDSP, PLA, Sc/Vi).
|
||||
- [x] **New game coverage**: Sword/Shield (1052 routes, 8900 encounters), Scarlet/Violet (415 routes, ~3850 encounters), Legends Arceus (88 routes, 1582 encounters), BDSP (177 routes, ~1840 encounters) — all reasonable.
|
||||
- [ ] **Update route_order.json**: 11 version groups have route orders, 11 are missing (diamond-pearl, black-white, black-2-white-2, x-y, sun-moon, ultra-sun-ultra-moon, sword-shield, brilliant-diamond-shining-pearl, legends-arceus, scarlet-violet). Requires manual curation.
|
||||
- [ ] **Update special_encounters.json**: Currently only covers 3 version groups (firered-leafgreen, heartgold-soulsilver, emerald + aliases). Requires manual curation for other games.
|
||||
|
||||
## Notes
|
||||
- This is the final validation step before we can replace PokeAPI as the data source
|
||||
- Some discrepancies are expected — PokeDB may have more complete data than PokeAPI
|
||||
- Route ordering for new games will likely need manual work
|
||||
@@ -0,0 +1,26 @@
|
||||
---
|
||||
# nuzlocke-tracker-zno2
|
||||
title: Build reference data mappings
|
||||
status: completed
|
||||
type: task
|
||||
priority: normal
|
||||
created_at: 2026-02-11T08:43:02Z
|
||||
updated_at: 2026-02-11T09:03:01Z
|
||||
parent: nuzlocke-tracker-bs05
|
||||
blocking:
|
||||
- nuzlocke-tracker-rfg0
|
||||
---
|
||||
|
||||
Build the lookup maps needed to translate PokeDB identifiers into our seed format.
|
||||
|
||||
## Checklist
|
||||
|
||||
- [x] **Pokemon form mapping**: Map `pokemon_form_identifier` (e.g. "pidgey-default", "mr-mime-default") to `pokeapi_id` using the existing `backend/src/app/seeds/data/pokemon.json` as reference. Handle naming convention differences between PokeDB and PokeAPI (may need fuzzy matching or a manual override table).
|
||||
- [x] **Location area mapping**: Map `location_area_identifier` to human-readable location names and regions using `locations.json` and `location_areas.json`. Produce names matching our existing format (e.g. "Route 1", "Viridian Forest").
|
||||
- [x] **Encounter method mapping**: Map PokeDB's 73 encounter methods to our simplified set. See the draft mapping in the parent bean. Implement as a dictionary/config that's easy to extend.
|
||||
- [x] **Version mapping**: Map PokeDB `version_identifiers` to our game slugs (should mostly be 1:1 but verify).
|
||||
|
||||
## Notes
|
||||
- The pokemon form mapping is the trickiest part — PokeDB uses identifiers like "mr-mime-default" while our pokemon.json uses names like "Mr. Mime" and pokeapi IDs
|
||||
- Log warnings for any unmapped identifiers so we can add overrides
|
||||
- The `pokemon_forms.json` from PokeDB may help bridge the gap
|
||||
+5
-1
@@ -60,8 +60,12 @@ temp/
|
||||
*.tmp
|
||||
*.temp
|
||||
|
||||
# PokeAPI fetch cache
|
||||
# PokeAPI / PokeDB data cache
|
||||
.pokeapi_cache/
|
||||
.pokedb_cache/
|
||||
|
||||
# Generated sprites (downloaded by import-pokedb tool)
|
||||
frontend/public/sprites/
|
||||
|
||||
# Go build output
|
||||
tools/fetch-pokeapi/fetch-pokeapi
|
||||
|
||||
File diff suppressed because it is too large
Load Diff
+8467
-4616
File diff suppressed because it is too large
Load Diff
+5940
-3314
File diff suppressed because it is too large
Load Diff
+2208
-1894
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+10641
-9334
File diff suppressed because it is too large
Load Diff
+7683
-8349
File diff suppressed because it is too large
Load Diff
+4759
-4280
File diff suppressed because it is too large
Load Diff
+2916
-3121
File diff suppressed because it is too large
Load Diff
@@ -1,344 +1,344 @@
|
||||
[
|
||||
{
|
||||
"name": "Pokemon Alpha Sapphire",
|
||||
"slug": "alpha-sapphire",
|
||||
"generation": 6,
|
||||
"region": "hoenn",
|
||||
"category": "remake",
|
||||
"release_year": 2014,
|
||||
"color": "#26649C"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Black",
|
||||
"slug": "black",
|
||||
"generation": 5,
|
||||
"region": "unova",
|
||||
"category": "original",
|
||||
"release_year": 2010,
|
||||
"color": "#444444"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Black 2",
|
||||
"slug": "black-2",
|
||||
"generation": 5,
|
||||
"region": "unova",
|
||||
"category": "sequel",
|
||||
"release_year": 2012,
|
||||
"color": "#424B50"
|
||||
"name": "Pokemon Red",
|
||||
"slug": "red",
|
||||
"generation": 1,
|
||||
"region": "kanto",
|
||||
"release_year": 1996,
|
||||
"color": "#FF1111",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Blue",
|
||||
"slug": "blue",
|
||||
"generation": 1,
|
||||
"region": "kanto",
|
||||
"category": "original",
|
||||
"release_year": 1996,
|
||||
"color": "#1111FF"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Brilliant Diamond",
|
||||
"slug": "brilliant-diamond",
|
||||
"generation": 8,
|
||||
"region": "sinnoh",
|
||||
"category": "remake",
|
||||
"release_year": 2021,
|
||||
"color": "#44BAE5"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Crystal",
|
||||
"slug": "crystal",
|
||||
"generation": 2,
|
||||
"region": "johto",
|
||||
"category": "enhanced",
|
||||
"release_year": 2000,
|
||||
"color": "#4FD9FF"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Diamond",
|
||||
"slug": "diamond",
|
||||
"generation": 4,
|
||||
"region": "sinnoh",
|
||||
"category": "original",
|
||||
"release_year": 2006,
|
||||
"color": "#AAAAFF"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Emerald",
|
||||
"slug": "emerald",
|
||||
"generation": 3,
|
||||
"region": "hoenn",
|
||||
"category": "enhanced",
|
||||
"release_year": 2005,
|
||||
"color": "#00A000"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon FireRed",
|
||||
"slug": "firered",
|
||||
"generation": 3,
|
||||
"region": "kanto",
|
||||
"category": "remake",
|
||||
"release_year": 2004,
|
||||
"color": "#FF7327"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Gold",
|
||||
"slug": "gold",
|
||||
"generation": 2,
|
||||
"region": "johto",
|
||||
"category": "original",
|
||||
"release_year": 1999,
|
||||
"color": "#DAA520"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon HeartGold",
|
||||
"slug": "heartgold",
|
||||
"generation": 4,
|
||||
"region": "johto",
|
||||
"category": "remake",
|
||||
"release_year": 2010,
|
||||
"color": "#B69E00"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon LeafGreen",
|
||||
"slug": "leafgreen",
|
||||
"generation": 3,
|
||||
"region": "kanto",
|
||||
"category": "remake",
|
||||
"release_year": 2004,
|
||||
"color": "#00DD00"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Legends: Arceus",
|
||||
"slug": "legends-arceus",
|
||||
"generation": 8,
|
||||
"region": "hisui",
|
||||
"category": "spinoff",
|
||||
"release_year": 2022,
|
||||
"color": "#36597B"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Legends: Z-A",
|
||||
"slug": "legends-z-a",
|
||||
"generation": 9,
|
||||
"region": "lumiose",
|
||||
"category": "spinoff",
|
||||
"release_year": 2025,
|
||||
"color": "#3A7BDB"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Let's Go Eevee",
|
||||
"slug": "lets-go-eevee",
|
||||
"generation": 7,
|
||||
"region": "kanto",
|
||||
"category": "remake",
|
||||
"release_year": 2018,
|
||||
"color": "#D4924B"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Let's Go Pikachu",
|
||||
"slug": "lets-go-pikachu",
|
||||
"generation": 7,
|
||||
"region": "kanto",
|
||||
"category": "remake",
|
||||
"release_year": 2018,
|
||||
"color": "#F5DA00"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Moon",
|
||||
"slug": "moon",
|
||||
"generation": 7,
|
||||
"region": "alola",
|
||||
"category": "original",
|
||||
"release_year": 2016,
|
||||
"color": "#5599CA"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Omega Ruby",
|
||||
"slug": "omega-ruby",
|
||||
"generation": 6,
|
||||
"region": "hoenn",
|
||||
"category": "remake",
|
||||
"release_year": 2014,
|
||||
"color": "#CF3025"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Pearl",
|
||||
"slug": "pearl",
|
||||
"generation": 4,
|
||||
"region": "sinnoh",
|
||||
"category": "original",
|
||||
"release_year": 2006,
|
||||
"color": "#FFAAAA"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Platinum",
|
||||
"slug": "platinum",
|
||||
"generation": 4,
|
||||
"region": "sinnoh",
|
||||
"category": "enhanced",
|
||||
"release_year": 2008,
|
||||
"color": "#999999"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Red",
|
||||
"slug": "red",
|
||||
"generation": 1,
|
||||
"region": "kanto",
|
||||
"category": "original",
|
||||
"release_year": 1996,
|
||||
"color": "#FF1111"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Ruby",
|
||||
"slug": "ruby",
|
||||
"generation": 3,
|
||||
"region": "hoenn",
|
||||
"category": "original",
|
||||
"release_year": 2002,
|
||||
"color": "#A00000"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Sapphire",
|
||||
"slug": "sapphire",
|
||||
"generation": 3,
|
||||
"region": "hoenn",
|
||||
"category": "original",
|
||||
"release_year": 2002,
|
||||
"color": "#0000A0"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Scarlet",
|
||||
"slug": "scarlet",
|
||||
"generation": 9,
|
||||
"region": "paldea",
|
||||
"category": "original",
|
||||
"release_year": 2022,
|
||||
"color": "#F93C3C"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Shield",
|
||||
"slug": "shield",
|
||||
"generation": 8,
|
||||
"region": "galar",
|
||||
"category": "original",
|
||||
"release_year": 2019,
|
||||
"color": "#EF3B6E"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Shining Pearl",
|
||||
"slug": "shining-pearl",
|
||||
"generation": 8,
|
||||
"region": "sinnoh",
|
||||
"category": "remake",
|
||||
"release_year": 2021,
|
||||
"color": "#E18AAA"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Silver",
|
||||
"slug": "silver",
|
||||
"generation": 2,
|
||||
"region": "johto",
|
||||
"category": "original",
|
||||
"release_year": 1999,
|
||||
"color": "#C0C0C0"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon SoulSilver",
|
||||
"slug": "soulsilver",
|
||||
"generation": 4,
|
||||
"region": "johto",
|
||||
"category": "remake",
|
||||
"release_year": 2010,
|
||||
"color": "#C0C0E0"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Sun",
|
||||
"slug": "sun",
|
||||
"generation": 7,
|
||||
"region": "alola",
|
||||
"category": "original",
|
||||
"release_year": 2016,
|
||||
"color": "#F1912B"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Sword",
|
||||
"slug": "sword",
|
||||
"generation": 8,
|
||||
"region": "galar",
|
||||
"category": "original",
|
||||
"release_year": 2019,
|
||||
"color": "#00D4E7"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Ultra Moon",
|
||||
"slug": "ultra-moon",
|
||||
"generation": 7,
|
||||
"region": "alola",
|
||||
"category": "enhanced",
|
||||
"release_year": 2017,
|
||||
"color": "#204E8C"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Ultra Sun",
|
||||
"slug": "ultra-sun",
|
||||
"generation": 7,
|
||||
"region": "alola",
|
||||
"category": "enhanced",
|
||||
"release_year": 2017,
|
||||
"color": "#E95B2B"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Violet",
|
||||
"slug": "violet",
|
||||
"generation": 9,
|
||||
"region": "paldea",
|
||||
"category": "original",
|
||||
"release_year": 2022,
|
||||
"color": "#A96EEC"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon White",
|
||||
"slug": "white",
|
||||
"generation": 5,
|
||||
"region": "unova",
|
||||
"category": "original",
|
||||
"release_year": 2010,
|
||||
"color": "#E1E1E1"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon White 2",
|
||||
"slug": "white-2",
|
||||
"generation": 5,
|
||||
"region": "unova",
|
||||
"category": "sequel",
|
||||
"release_year": 2012,
|
||||
"color": "#E3CED0"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon X",
|
||||
"slug": "x",
|
||||
"generation": 6,
|
||||
"region": "kalos",
|
||||
"category": "original",
|
||||
"release_year": 2013,
|
||||
"color": "#025DA6"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Y",
|
||||
"slug": "y",
|
||||
"generation": 6,
|
||||
"region": "kalos",
|
||||
"category": "original",
|
||||
"release_year": 2013,
|
||||
"color": "#EA1A3E"
|
||||
"color": "#1111FF",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Yellow",
|
||||
"slug": "yellow",
|
||||
"generation": 1,
|
||||
"region": "kanto",
|
||||
"category": "enhanced",
|
||||
"release_year": 1998,
|
||||
"color": "#FFD733"
|
||||
"color": "#FFD733",
|
||||
"category": "enhanced"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Gold",
|
||||
"slug": "gold",
|
||||
"generation": 2,
|
||||
"region": "johto",
|
||||
"release_year": 1999,
|
||||
"color": "#DAA520",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Silver",
|
||||
"slug": "silver",
|
||||
"generation": 2,
|
||||
"region": "johto",
|
||||
"release_year": 1999,
|
||||
"color": "#C0C0C0",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Crystal",
|
||||
"slug": "crystal",
|
||||
"generation": 2,
|
||||
"region": "johto",
|
||||
"release_year": 2000,
|
||||
"color": "#4FD9FF",
|
||||
"category": "enhanced"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Ruby",
|
||||
"slug": "ruby",
|
||||
"generation": 3,
|
||||
"region": "hoenn",
|
||||
"release_year": 2002,
|
||||
"color": "#A00000",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Sapphire",
|
||||
"slug": "sapphire",
|
||||
"generation": 3,
|
||||
"region": "hoenn",
|
||||
"release_year": 2002,
|
||||
"color": "#0000A0",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Emerald",
|
||||
"slug": "emerald",
|
||||
"generation": 3,
|
||||
"region": "hoenn",
|
||||
"release_year": 2005,
|
||||
"color": "#00A000",
|
||||
"category": "enhanced"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon FireRed",
|
||||
"slug": "firered",
|
||||
"generation": 3,
|
||||
"region": "kanto",
|
||||
"release_year": 2004,
|
||||
"color": "#FF7327",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon LeafGreen",
|
||||
"slug": "leafgreen",
|
||||
"generation": 3,
|
||||
"region": "kanto",
|
||||
"release_year": 2004,
|
||||
"color": "#00DD00",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Diamond",
|
||||
"slug": "diamond",
|
||||
"generation": 4,
|
||||
"region": "sinnoh",
|
||||
"release_year": 2006,
|
||||
"color": "#AAAAFF",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Pearl",
|
||||
"slug": "pearl",
|
||||
"generation": 4,
|
||||
"region": "sinnoh",
|
||||
"release_year": 2006,
|
||||
"color": "#FFAAAA",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Platinum",
|
||||
"slug": "platinum",
|
||||
"generation": 4,
|
||||
"region": "sinnoh",
|
||||
"release_year": 2008,
|
||||
"color": "#999999",
|
||||
"category": "enhanced"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon HeartGold",
|
||||
"slug": "heartgold",
|
||||
"generation": 4,
|
||||
"region": "johto",
|
||||
"release_year": 2010,
|
||||
"color": "#B69E00",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon SoulSilver",
|
||||
"slug": "soulsilver",
|
||||
"generation": 4,
|
||||
"region": "johto",
|
||||
"release_year": 2010,
|
||||
"color": "#C0C0E0",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Black",
|
||||
"slug": "black",
|
||||
"generation": 5,
|
||||
"region": "unova",
|
||||
"release_year": 2010,
|
||||
"color": "#444444",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon White",
|
||||
"slug": "white",
|
||||
"generation": 5,
|
||||
"region": "unova",
|
||||
"release_year": 2010,
|
||||
"color": "#E1E1E1",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Black 2",
|
||||
"slug": "black-2",
|
||||
"generation": 5,
|
||||
"region": "unova",
|
||||
"release_year": 2012,
|
||||
"color": "#424B50",
|
||||
"category": "sequel"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon White 2",
|
||||
"slug": "white-2",
|
||||
"generation": 5,
|
||||
"region": "unova",
|
||||
"release_year": 2012,
|
||||
"color": "#E3CED0",
|
||||
"category": "sequel"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon X",
|
||||
"slug": "x",
|
||||
"generation": 6,
|
||||
"region": "kalos",
|
||||
"release_year": 2013,
|
||||
"color": "#025DA6",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Y",
|
||||
"slug": "y",
|
||||
"generation": 6,
|
||||
"region": "kalos",
|
||||
"release_year": 2013,
|
||||
"color": "#EA1A3E",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Omega Ruby",
|
||||
"slug": "omega-ruby",
|
||||
"generation": 6,
|
||||
"region": "hoenn",
|
||||
"release_year": 2014,
|
||||
"color": "#CF3025",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Alpha Sapphire",
|
||||
"slug": "alpha-sapphire",
|
||||
"generation": 6,
|
||||
"region": "hoenn",
|
||||
"release_year": 2014,
|
||||
"color": "#26649C",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Sun",
|
||||
"slug": "sun",
|
||||
"generation": 7,
|
||||
"region": "alola",
|
||||
"release_year": 2016,
|
||||
"color": "#F1912B",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Moon",
|
||||
"slug": "moon",
|
||||
"generation": 7,
|
||||
"region": "alola",
|
||||
"release_year": 2016,
|
||||
"color": "#5599CA",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Ultra Sun",
|
||||
"slug": "ultra-sun",
|
||||
"generation": 7,
|
||||
"region": "alola",
|
||||
"release_year": 2017,
|
||||
"color": "#E95B2B",
|
||||
"category": "enhanced"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Ultra Moon",
|
||||
"slug": "ultra-moon",
|
||||
"generation": 7,
|
||||
"region": "alola",
|
||||
"release_year": 2017,
|
||||
"color": "#204E8C",
|
||||
"category": "enhanced"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Let's Go Pikachu",
|
||||
"slug": "lets-go-pikachu",
|
||||
"generation": 7,
|
||||
"region": "kanto",
|
||||
"release_year": 2018,
|
||||
"color": "#F5DA00",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Let's Go Eevee",
|
||||
"slug": "lets-go-eevee",
|
||||
"generation": 7,
|
||||
"region": "kanto",
|
||||
"release_year": 2018,
|
||||
"color": "#D4924B",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Sword",
|
||||
"slug": "sword",
|
||||
"generation": 8,
|
||||
"region": "galar",
|
||||
"release_year": 2019,
|
||||
"color": "#00D4E7",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Shield",
|
||||
"slug": "shield",
|
||||
"generation": 8,
|
||||
"region": "galar",
|
||||
"release_year": 2019,
|
||||
"color": "#EF3B6E",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Brilliant Diamond",
|
||||
"slug": "brilliant-diamond",
|
||||
"generation": 8,
|
||||
"region": "sinnoh",
|
||||
"release_year": 2021,
|
||||
"color": "#44BAE5",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Shining Pearl",
|
||||
"slug": "shining-pearl",
|
||||
"generation": 8,
|
||||
"region": "sinnoh",
|
||||
"release_year": 2021,
|
||||
"color": "#E18AAA",
|
||||
"category": "remake"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Legends: Arceus",
|
||||
"slug": "legends-arceus",
|
||||
"generation": 8,
|
||||
"region": "hisui",
|
||||
"release_year": 2022,
|
||||
"color": "#36597B",
|
||||
"category": "spinoff"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Scarlet",
|
||||
"slug": "scarlet",
|
||||
"generation": 9,
|
||||
"region": "paldea",
|
||||
"release_year": 2022,
|
||||
"color": "#F93C3C",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Violet",
|
||||
"slug": "violet",
|
||||
"generation": 9,
|
||||
"region": "paldea",
|
||||
"release_year": 2022,
|
||||
"color": "#A96EEC",
|
||||
"category": "original"
|
||||
},
|
||||
{
|
||||
"name": "Pokemon Legends: Z-A",
|
||||
"slug": "legends-z-a",
|
||||
"generation": 9,
|
||||
"region": "lumiose",
|
||||
"release_year": 2025,
|
||||
"color": "#3A7BDB",
|
||||
"category": "spinoff"
|
||||
}
|
||||
]
|
||||
|
||||
+9732
-8363
File diff suppressed because it is too large
Load Diff
+17852
-14947
File diff suppressed because it is too large
Load Diff
+2942
-3147
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+4281
-3264
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+7870
-8511
File diff suppressed because it is too large
Load Diff
+5643
-6130
File diff suppressed because it is too large
Load Diff
+1375
-1387
File diff suppressed because it is too large
Load Diff
+2212
-1898
File diff suppressed because it is too large
Load Diff
+4668
-4294
File diff suppressed because it is too large
Load Diff
+4667
-4301
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+4771
-3426
File diff suppressed because it is too large
Load Diff
+9638
-6717
File diff suppressed because it is too large
Load Diff
+4388
-3348
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+5543
-3673
File diff suppressed because it is too large
Load Diff
+5534
-3672
File diff suppressed because it is too large
Load Diff
File diff suppressed because it is too large
Load Diff
+8459
-4608
File diff suppressed because it is too large
Load Diff
+6202
-3306
File diff suppressed because it is too large
Load Diff
+6020
-3110
File diff suppressed because it is too large
Load Diff
+6008
-3090
File diff suppressed because it is too large
Load Diff
+3664
-3334
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,239 @@
|
||||
"""CLI entry point for the PokeDB import tool.
|
||||
|
||||
Usage:
|
||||
# From tools/import-pokedb/ (auto-downloads PokeDB data on first run):
|
||||
python -m import_pokedb
|
||||
|
||||
# With options:
|
||||
python -m import_pokedb --game firered
|
||||
python -m import_pokedb --pokedb-dir ~/my-pokedb-data/
|
||||
python -m import_pokedb --output /tmp/seed-output/
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
from .loader import load_pokedb_data, load_seed_config
|
||||
from .mappings import PokemonMapper, LocationMapper, build_version_map, map_encounter_method
|
||||
from .output import sort_routes, merge_special_encounters, write_game_json, write_games_json, write_pokemon_json
|
||||
from .processing import filter_encounters_for_game, process_encounters, build_routes
|
||||
from .sprites import download_all_sprites, download_sprites
|
||||
|
||||
SEEDS_DIR_CANDIDATES = [
|
||||
Path("backend/src/app/seeds"), # from repo root
|
||||
Path("../../backend/src/app/seeds"), # from tools/import-pokedb/
|
||||
]
|
||||
|
||||
SPRITES_DIR_CANDIDATES = [
|
||||
Path("frontend/public/sprites"), # from repo root
|
||||
Path("../../frontend/public/sprites"), # from tools/import-pokedb/
|
||||
]
|
||||
|
||||
|
||||
def find_seeds_dir() -> Path:
|
||||
"""Locate the backend seeds directory."""
|
||||
for candidate in SEEDS_DIR_CANDIDATES:
|
||||
if (candidate / "version_groups.json").exists():
|
||||
return candidate.resolve()
|
||||
# Fallback
|
||||
return Path("backend/src/app/seeds").resolve()
|
||||
|
||||
|
||||
def find_sprites_dir() -> Path:
|
||||
"""Locate the frontend sprites directory."""
|
||||
for candidate in SPRITES_DIR_CANDIDATES:
|
||||
if candidate.parent.exists():
|
||||
return candidate.resolve()
|
||||
# Fallback
|
||||
return Path("frontend/public/sprites").resolve()
|
||||
|
||||
|
||||
def build_parser() -> argparse.ArgumentParser:
|
||||
parser = argparse.ArgumentParser(
|
||||
prog="import-pokedb",
|
||||
description="Convert PokeDB.org JSON data exports into nuzlocke-tracker seed format.",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--pokedb-dir",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Path to directory containing PokeDB JSON export files (default: .pokedb_cache/)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--output",
|
||||
type=Path,
|
||||
default=None,
|
||||
help="Output directory for seed JSON files (default: backend/src/app/seeds/data/)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--game",
|
||||
type=str,
|
||||
default=None,
|
||||
help="Generate data for a specific game slug only (default: all games)",
|
||||
)
|
||||
return parser
|
||||
|
||||
|
||||
def main(argv: list[str] | None = None) -> None:
|
||||
parser = build_parser()
|
||||
args = parser.parse_args(argv)
|
||||
|
||||
seeds_dir = find_seeds_dir()
|
||||
pokedb_dir: Path = args.pokedb_dir or (seeds_dir / ".pokedb_cache")
|
||||
output_dir: Path = args.output or (seeds_dir / "data")
|
||||
output_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
print(f"PokeDB data: {pokedb_dir.resolve()}")
|
||||
print(f"Seeds config: {seeds_dir}")
|
||||
print(f"Output: {output_dir.resolve()}")
|
||||
print()
|
||||
|
||||
# Load PokeDB export data
|
||||
pokedb = load_pokedb_data(pokedb_dir)
|
||||
print(pokedb.summary())
|
||||
print()
|
||||
|
||||
# Load existing seed configuration
|
||||
config = load_seed_config(seeds_dir)
|
||||
print(f"Loaded {len(config.version_groups)} version groups")
|
||||
print(f"Loaded route order for {len(config.route_order)} version groups")
|
||||
if config.special_encounters:
|
||||
se_count = len(config.special_encounters.get("encounters", {}))
|
||||
print(f"Loaded special encounters for {se_count} version groups")
|
||||
print()
|
||||
|
||||
# Determine which games to process
|
||||
target_game = args.game
|
||||
if target_game:
|
||||
found = False
|
||||
for vg_info in config.version_groups.values():
|
||||
if target_game in vg_info.get("versions", []):
|
||||
found = True
|
||||
break
|
||||
if not found:
|
||||
print(f"Error: Game '{target_game}' not found in version_groups.json", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
print(f"Target: {target_game}")
|
||||
else:
|
||||
total_games = sum(
|
||||
len(vg.get("versions", []))
|
||||
for vg in config.version_groups.values()
|
||||
)
|
||||
print(f"Target: all {total_games} games")
|
||||
|
||||
# Build mappings
|
||||
print("\nBuilding mappings...")
|
||||
|
||||
pokemon_json = seeds_dir / "data" / "pokemon.json"
|
||||
pokemon_mapper = PokemonMapper(pokemon_json, pokedb)
|
||||
|
||||
location_mapper = LocationMapper(pokedb)
|
||||
|
||||
version_map = build_version_map(pokedb, config.version_groups)
|
||||
print(f" Mapped {len(version_map)} PokeDB versions to our game slugs")
|
||||
|
||||
# Report encounter method coverage
|
||||
pokedb_methods = {e.get("encounter_method_identifier", "") for e in pokedb.encounters}
|
||||
pokedb_methods.discard("")
|
||||
mapped_methods = {m for m in pokedb_methods if map_encounter_method(m) is not None}
|
||||
unmapped_methods = pokedb_methods - mapped_methods
|
||||
print(f" Encounter methods: {len(mapped_methods)} mapped, {len(unmapped_methods)} unmapped")
|
||||
if unmapped_methods:
|
||||
print(" Unmapped methods:", file=sys.stderr)
|
||||
for m in sorted(unmapped_methods):
|
||||
print(f" - {m}", file=sys.stderr)
|
||||
|
||||
# Spot-check pokemon mapping on actual encounter data
|
||||
form_ids_in_encounters: set[str] = set()
|
||||
for e in pokedb.encounters:
|
||||
fid = e.get("pokemon_form_identifier")
|
||||
if fid:
|
||||
form_ids_in_encounters.add(fid)
|
||||
mapped_forms = 0
|
||||
for fid in form_ids_in_encounters:
|
||||
if pokemon_mapper.lookup(fid) is not None:
|
||||
mapped_forms += 1
|
||||
total_forms = len(form_ids_in_encounters)
|
||||
print(f" Pokemon forms: {mapped_forms}/{total_forms} mapped from encounters")
|
||||
|
||||
pokemon_mapper.report_unmapped()
|
||||
|
||||
# Process encounters per game
|
||||
print("\nProcessing encounters...")
|
||||
|
||||
games_to_process: list[tuple[str, str, int]] = [] # (vg_key, game_slug, generation)
|
||||
for vg_key, vg_info in config.version_groups.items():
|
||||
generation = vg_info.get("generation", 0)
|
||||
for slug in vg_info.get("versions", []):
|
||||
if target_game and slug != target_game:
|
||||
continue
|
||||
games_to_process.append((vg_key, slug, generation))
|
||||
|
||||
all_encountered_form_ids: set[str] = set()
|
||||
|
||||
for vg_key, game_slug, generation in games_to_process:
|
||||
print(f"\n--- {game_slug} ---")
|
||||
|
||||
# Filter encounters for this game
|
||||
game_encounters = filter_encounters_for_game(pokedb.encounters, game_slug)
|
||||
if not game_encounters:
|
||||
print(f" No encounters found in PokeDB data")
|
||||
continue
|
||||
|
||||
print(f" Raw encounters: {len(game_encounters)}")
|
||||
|
||||
# Track encountered form IDs for pokemon.json
|
||||
for e in game_encounters:
|
||||
fid = e.get("pokemon_form_identifier")
|
||||
if fid:
|
||||
all_encountered_form_ids.add(fid)
|
||||
|
||||
# Process into grouped encounters
|
||||
encounters_by_area = process_encounters(
|
||||
game_encounters, generation, pokemon_mapper, location_mapper,
|
||||
)
|
||||
print(f" Location areas with encounters: {len(encounters_by_area)}")
|
||||
|
||||
# Build route hierarchy
|
||||
routes = build_routes(encounters_by_area, location_mapper)
|
||||
|
||||
# Merge special encounters (starters, gifts, fossils)
|
||||
routes = merge_special_encounters(routes, config, vg_key, pokemon_mapper)
|
||||
|
||||
# Sort routes by game progression order
|
||||
routes = sort_routes(routes, config, vg_key)
|
||||
|
||||
# Stats
|
||||
total_routes = sum(1 + len(r.children) for r in routes)
|
||||
total_enc = sum(
|
||||
len(r.encounters) + sum(len(c.encounters) for c in r.children)
|
||||
for r in routes
|
||||
)
|
||||
print(f" Routes: {total_routes}")
|
||||
print(f" Encounter entries: {total_enc}")
|
||||
|
||||
# Write per-game JSON
|
||||
write_game_json(routes, output_dir, game_slug)
|
||||
|
||||
# Download sprites to frontend/public/sprites
|
||||
sprites_dir = find_sprites_dir()
|
||||
print(f"\nDownloading sprites to {sprites_dir}...")
|
||||
sprite_map = download_sprites(pokemon_mapper, all_encountered_form_ids, sprites_dir)
|
||||
print(f" Sprite map covers {len(sprite_map)} forms")
|
||||
|
||||
# Download sprites for ALL pokemon (including non-encountered evolutions etc.)
|
||||
all_sprite_ids = download_all_sprites(pokemon_mapper, sprites_dir)
|
||||
|
||||
# Write global JSON files
|
||||
print("\nWriting global data files...")
|
||||
write_games_json(config, output_dir)
|
||||
write_pokemon_json(pokemon_mapper, all_encountered_form_ids, sprite_map, output_dir)
|
||||
|
||||
print("\nDone!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,168 @@
|
||||
"""Load and validate PokeDB JSON export files."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import sys
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
_CDN_BASE = "https://cdn.pokedb.org"
|
||||
|
||||
REQUIRED_FILES: dict[str, str] = {
|
||||
"encounters.json": f"{_CDN_BASE}/data_export_encounters_json",
|
||||
"locations.json": f"{_CDN_BASE}/data_export_locations_json",
|
||||
"location_areas.json": f"{_CDN_BASE}/data_export_location_areas_json",
|
||||
"encounter_methods.json": f"{_CDN_BASE}/data_export_encounter_methods_json",
|
||||
"versions.json": f"{_CDN_BASE}/data_export_versions_json",
|
||||
"pokemon_forms.json": f"{_CDN_BASE}/data_export_pokemon_forms_json",
|
||||
}
|
||||
|
||||
|
||||
class PokeDBData:
|
||||
"""Container for all loaded PokeDB export data."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
encounters: list[dict[str, Any]],
|
||||
locations: list[dict[str, Any]],
|
||||
location_areas: list[dict[str, Any]],
|
||||
encounter_methods: list[dict[str, Any]],
|
||||
versions: list[dict[str, Any]],
|
||||
pokemon_forms: list[dict[str, Any]],
|
||||
) -> None:
|
||||
self.encounters = encounters
|
||||
self.locations = locations
|
||||
self.location_areas = location_areas
|
||||
self.encounter_methods = encounter_methods
|
||||
self.versions = versions
|
||||
self.pokemon_forms = pokemon_forms
|
||||
|
||||
def summary(self) -> str:
|
||||
return (
|
||||
f"PokeDB data loaded:\n"
|
||||
f" encounters: {len(self.encounters):,}\n"
|
||||
f" locations: {len(self.locations):,}\n"
|
||||
f" location_areas: {len(self.location_areas):,}\n"
|
||||
f" encounter_methods: {len(self.encounter_methods):,}\n"
|
||||
f" versions: {len(self.versions):,}\n"
|
||||
f" pokemon_forms: {len(self.pokemon_forms):,}"
|
||||
)
|
||||
|
||||
|
||||
def download_pokedb_data(data_dir: Path) -> None:
|
||||
"""Download missing PokeDB JSON export files into data_dir."""
|
||||
data_dir.mkdir(parents=True, exist_ok=True)
|
||||
missing = {f: url for f, url in REQUIRED_FILES.items() if not (data_dir / f).exists()}
|
||||
if not missing:
|
||||
return
|
||||
|
||||
print(f"Downloading {len(missing)} PokeDB file(s) to {data_dir}...")
|
||||
for filename, url in missing.items():
|
||||
dest = data_dir / filename
|
||||
print(f" {filename}...", end="", flush=True)
|
||||
try:
|
||||
urllib.request.urlretrieve(url, dest)
|
||||
size_mb = dest.stat().st_size / (1024 * 1024)
|
||||
print(f" {size_mb:.1f} MB")
|
||||
except Exception as e:
|
||||
print(f" FAILED", file=sys.stderr)
|
||||
print(f"Error downloading {url}: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
print()
|
||||
|
||||
|
||||
def load_pokedb_data(data_dir: Path) -> PokeDBData:
|
||||
"""Load all PokeDB JSON export files from a directory.
|
||||
|
||||
Downloads any missing files automatically, then validates and loads them.
|
||||
"""
|
||||
download_pokedb_data(data_dir)
|
||||
|
||||
missing = [f for f in REQUIRED_FILES if not (data_dir / f).exists()]
|
||||
if missing:
|
||||
print(f"Error: Still missing files after download: {missing}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
def _load(filename: str) -> list[dict[str, Any]]:
|
||||
path = data_dir / filename
|
||||
try:
|
||||
with open(path) as f:
|
||||
data = json.load(f)
|
||||
except json.JSONDecodeError as e:
|
||||
print(f"Error: Failed to parse {path}: {e}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
if not isinstance(data, list):
|
||||
print(
|
||||
f"Error: Expected a JSON array in {path}, got {type(data).__name__}",
|
||||
file=sys.stderr,
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
return data
|
||||
|
||||
return PokeDBData(
|
||||
encounters=_load("encounters.json"),
|
||||
locations=_load("locations.json"),
|
||||
location_areas=_load("location_areas.json"),
|
||||
encounter_methods=_load("encounter_methods.json"),
|
||||
versions=_load("versions.json"),
|
||||
pokemon_forms=_load("pokemon_forms.json"),
|
||||
)
|
||||
|
||||
|
||||
class SeedConfig:
|
||||
"""Container for existing seed configuration files."""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
version_groups: dict[str, Any],
|
||||
route_order: dict[str, list[str]],
|
||||
special_encounters: dict[str, Any] | None,
|
||||
) -> None:
|
||||
self.version_groups = version_groups
|
||||
self.route_order = route_order
|
||||
self.special_encounters = special_encounters
|
||||
|
||||
|
||||
def load_seed_config(seeds_dir: Path) -> SeedConfig:
|
||||
"""Load existing seed configuration files (version_groups, route_order, etc.).
|
||||
|
||||
Exits with an error message if required config files are missing.
|
||||
"""
|
||||
vg_path = seeds_dir / "version_groups.json"
|
||||
if not vg_path.exists():
|
||||
print(f"Error: version_groups.json not found at {vg_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
with open(vg_path) as f:
|
||||
version_groups = json.load(f)
|
||||
|
||||
# Load route_order.json and resolve aliases
|
||||
ro_path = seeds_dir / "route_order.json"
|
||||
if not ro_path.exists():
|
||||
print(f"Error: route_order.json not found at {ro_path}", file=sys.stderr)
|
||||
sys.exit(1)
|
||||
|
||||
with open(ro_path) as f:
|
||||
ro_raw = json.load(f)
|
||||
|
||||
route_order: dict[str, list[str]] = dict(ro_raw.get("routes", {}))
|
||||
for alias, target in ro_raw.get("aliases", {}).items():
|
||||
if target in route_order:
|
||||
route_order[alias] = route_order[target]
|
||||
|
||||
# Load special_encounters.json (optional)
|
||||
se_path = seeds_dir / "special_encounters.json"
|
||||
special_encounters = None
|
||||
if se_path.exists():
|
||||
with open(se_path) as f:
|
||||
special_encounters = json.load(f)
|
||||
|
||||
return SeedConfig(
|
||||
version_groups=version_groups,
|
||||
route_order=route_order,
|
||||
special_encounters=special_encounters,
|
||||
)
|
||||
@@ -0,0 +1,683 @@
|
||||
"""Reference data mappings: PokeDB identifiers → seed format values."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .loader import PokeDBData
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Encounter method mapping
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# PokeDB encounter_method_identifier → our simplified method name.
|
||||
# Keys can be exact matches or prefix patterns (ending with *).
|
||||
ENCOUNTER_METHOD_MAP: dict[str, str] = {
|
||||
# Walking / grass / cave
|
||||
"walking-tall-grass": "walk",
|
||||
"walking-long-grass": "walk",
|
||||
"walking-cave": "walk",
|
||||
"walking-bridge": "walk",
|
||||
"walking-building": "walk",
|
||||
"walking-sand": "walk",
|
||||
"walking-snow": "walk",
|
||||
"walking-rough-terrain": "walk",
|
||||
"walking-marsh": "walk",
|
||||
"walking-puddle": "walk",
|
||||
"walking-flower-field": "walk",
|
||||
"walking-ice": "walk",
|
||||
"walking-forest": "walk",
|
||||
"walking-snowfield": "walk",
|
||||
"dark-grass": "walk",
|
||||
"shaking-grass": "walk",
|
||||
"rustling-grass": "walk",
|
||||
"yellow-flowers": "walk",
|
||||
"red-flowers": "walk",
|
||||
"purple-flowers": "walk",
|
||||
# Surfing
|
||||
"surfing": "surf",
|
||||
"surfing-ocean": "surf",
|
||||
"surfing-puddle": "surf",
|
||||
"surfing-rapids": "surf",
|
||||
"surfing-underwater": "surf",
|
||||
"rippling-water": "surf",
|
||||
# Fishing
|
||||
"fishing-old-rod": "old-rod",
|
||||
"fishing-good-rod": "good-rod",
|
||||
"fishing-super-rod": "super-rod",
|
||||
"fishing": "fishing",
|
||||
"fishing-special": "fishing",
|
||||
# Rock smash
|
||||
"rock-smash": "rock-smash",
|
||||
# Headbutt
|
||||
"headbutt-low": "headbutt",
|
||||
"headbutt-normal": "headbutt",
|
||||
"headbutt-high": "headbutt",
|
||||
# Gift / special acquisition
|
||||
"npc-gift": "gift",
|
||||
"egg": "gift",
|
||||
"revive": "gift",
|
||||
"fossil": "gift",
|
||||
# Trade
|
||||
"npc-trade": "trade",
|
||||
# Overworld / symbol encounters (Gen 8+)
|
||||
"symbol-encounter": "walk",
|
||||
"wanderer": "walk",
|
||||
"flying": "walk",
|
||||
# Static / fixed
|
||||
"fixed-encounter": "static",
|
||||
"static-encounter": "static",
|
||||
"legendary-encounter": "static",
|
||||
"interactable": "static",
|
||||
# Special methods
|
||||
"swarm": "swarm",
|
||||
"poke-radar": "pokeradar",
|
||||
"dual-slot-mode": "dual-slot",
|
||||
"honey-tree": "honey",
|
||||
"trophy-garden": "walk",
|
||||
"great-marsh": "walk",
|
||||
"cave-spot": "walk",
|
||||
"bubble-spot": "surf",
|
||||
"sand-spot": "walk",
|
||||
"horde": "walk",
|
||||
"sos-encounter": "walk",
|
||||
"ambush": "walk",
|
||||
# Seaweed / diving
|
||||
"diving": "surf",
|
||||
"diving-seaweed": "surf",
|
||||
"seaweed": "surf",
|
||||
# Raids
|
||||
"max-raid": "raid",
|
||||
"max-raid-battle": "raid",
|
||||
"dynamax-adventure": "raid",
|
||||
"tera-raid": "raid",
|
||||
"tera-raid-battle": "raid",
|
||||
"fixed-tera-encounter": "static",
|
||||
# Misc
|
||||
"roaming": "roaming",
|
||||
"safari-zone": "walk",
|
||||
"bug-contest": "walk",
|
||||
"dust-cloud": "walk",
|
||||
"hidden-grotto": "static",
|
||||
"hidden-encounter": "walk",
|
||||
"horde-encounter": "walk",
|
||||
"shaking-trees": "walk",
|
||||
"shaking-ore-deposits": "walk",
|
||||
"island-scan": "static",
|
||||
"mass-outbreak": "swarm",
|
||||
"npc-buy": "gift",
|
||||
"special-encounter": "static",
|
||||
"sea-skim": "surf",
|
||||
"midair": "walk",
|
||||
"mr-backlot": "walk",
|
||||
"hoenn-sound": "walk",
|
||||
"sinnoh-sound": "walk",
|
||||
"curry": "gift",
|
||||
"boxes": "gift",
|
||||
"berry-tree": "walk",
|
||||
"zygarde-cube-assemble": "static",
|
||||
"contact-flock": "walk",
|
||||
"contact-space-time-distortion": "walk",
|
||||
"contact-unown-reasearch-notes": "static",
|
||||
"flying-pokemon-shadow": "walk",
|
||||
}
|
||||
|
||||
# Prefix-based fallbacks for methods not explicitly listed above.
|
||||
_METHOD_PREFIX_MAP: list[tuple[str, str]] = [
|
||||
("walking-", "walk"),
|
||||
("surfing-", "surf"),
|
||||
("fishing-", "fishing"),
|
||||
("headbutt-", "headbutt"),
|
||||
("flying-", "walk"),
|
||||
("ambush-", "walk"),
|
||||
("contact-", "walk"),
|
||||
]
|
||||
|
||||
|
||||
def map_encounter_method(method_identifier: str) -> str | None:
|
||||
"""Map a PokeDB encounter method to our simplified method name.
|
||||
|
||||
Returns None if the method is unrecognized.
|
||||
"""
|
||||
if method_identifier in ENCOUNTER_METHOD_MAP:
|
||||
return ENCOUNTER_METHOD_MAP[method_identifier]
|
||||
|
||||
for prefix, mapped in _METHOD_PREFIX_MAP:
|
||||
if method_identifier.startswith(prefix):
|
||||
return mapped
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Version mapping
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# PokeDB version identifiers that differ from our game slugs.
|
||||
# Most are 1:1, these handle exceptions.
|
||||
_VERSION_OVERRIDES: dict[str, str] = {
|
||||
"lets-go-pikachu": "lets-go-pikachu",
|
||||
"lets-go-eevee": "lets-go-eevee",
|
||||
}
|
||||
|
||||
|
||||
def build_version_map(
|
||||
pokedb: PokeDBData,
|
||||
version_groups: dict[str, Any],
|
||||
) -> dict[str, str]:
|
||||
"""Build a mapping from PokeDB version_identifier → our game slug.
|
||||
|
||||
Returns the mapping dict. Logs warnings for unmapped versions.
|
||||
"""
|
||||
# Collect all our known game slugs
|
||||
our_slugs: set[str] = set()
|
||||
for vg in version_groups.values():
|
||||
for slug in vg.get("versions", []):
|
||||
our_slugs.add(slug)
|
||||
|
||||
# Collect all PokeDB version identifiers
|
||||
pokedb_versions: set[str] = set()
|
||||
for v in pokedb.versions:
|
||||
identifier = v.get("identifier", "")
|
||||
if identifier:
|
||||
pokedb_versions.add(identifier)
|
||||
|
||||
mapping: dict[str, str] = {}
|
||||
|
||||
for pdb_ver in sorted(pokedb_versions):
|
||||
if pdb_ver in _VERSION_OVERRIDES:
|
||||
mapping[pdb_ver] = _VERSION_OVERRIDES[pdb_ver]
|
||||
elif pdb_ver in our_slugs:
|
||||
mapping[pdb_ver] = pdb_ver
|
||||
# else: PokeDB version not in our version_groups (expected for some)
|
||||
|
||||
# Report our games that have no PokeDB mapping
|
||||
mapped_slugs = set(mapping.values())
|
||||
unmapped_ours = our_slugs - mapped_slugs
|
||||
if unmapped_ours:
|
||||
print(f" Versions in our config with no PokeDB match: {sorted(unmapped_ours)}")
|
||||
|
||||
return mapping
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Pokemon form mapping
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
|
||||
# PokeDB uses adjectival region forms ("alolan") while PokeAPI/our data uses
|
||||
# region names ("alola"). This maps PokeDB suffixes → our suffixes.
|
||||
_FORM_SUFFIX_MAP: dict[str, str] = {
|
||||
"alolan": "alola",
|
||||
"galarian": "galar",
|
||||
"hisuian": "hisui",
|
||||
"paldean": "paldea",
|
||||
# Totem forms
|
||||
"alolan-totem": "totem-alola",
|
||||
# Basculin stripes
|
||||
"blue-stripe": "blue-striped",
|
||||
"red-stripe": "red-striped",
|
||||
"white-stripe": "white-striped",
|
||||
# Sea forms
|
||||
"west-sea": "west",
|
||||
"east-sea": "east",
|
||||
# Cloak forms
|
||||
"plant-cloak": "plant",
|
||||
"sandy-cloak": "sandy",
|
||||
"trash-cloak": "trash",
|
||||
# Eiscue
|
||||
"ice-face": "ice",
|
||||
# Misc forms
|
||||
"pompom": "pom-pom",
|
||||
"10p": "10",
|
||||
"50p": "50",
|
||||
"owntempo": "own-tempo",
|
||||
"two": "two-segment",
|
||||
"chest": "chest-form",
|
||||
"ice-rider": "ice",
|
||||
"shadow-rider": "shadow",
|
||||
"apex": "apex-build",
|
||||
"ultimate": "ultimate-mode",
|
||||
"black-activated": "black",
|
||||
"white-activated": "white",
|
||||
"hero": "hero",
|
||||
"sword": "crowned",
|
||||
"shield": "crowned",
|
||||
# Gigantamax
|
||||
"gigantamax": "gmax",
|
||||
# Partner forms
|
||||
"partner": "partner-cap",
|
||||
# Flabébé / Floette / Florges color forms — these don't have form suffixes in our data
|
||||
# since each color is just the base form. Map to base.
|
||||
"blue": "blue",
|
||||
"orange": "orange",
|
||||
"red": "red",
|
||||
"white": "white",
|
||||
"yellow": "yellow",
|
||||
# Gender forms
|
||||
"female": "female",
|
||||
"male": "male",
|
||||
# Furfrou
|
||||
"natural": "natural",
|
||||
# Cherrim
|
||||
"overcast": "overcast",
|
||||
# Sinistea / Polteageist
|
||||
"antique": "antique",
|
||||
"phony": "phony",
|
||||
# Poltchageist / Sinistcha
|
||||
"artisan": "artisan",
|
||||
"counterfeit": "counterfeit",
|
||||
"masterpiece": "masterpiece",
|
||||
"unremarkable": "unremarkable",
|
||||
# Minior cores
|
||||
"blue-core": "blue",
|
||||
"green-core": "green",
|
||||
"indigo-core": "indigo",
|
||||
"orange-core": "orange",
|
||||
"red-core": "red",
|
||||
"violet-core": "violet",
|
||||
"yellow-core": "yellow",
|
||||
# Vivillon
|
||||
"fancy": "fancy",
|
||||
# Squawkabilly
|
||||
# these use same name
|
||||
# Xerneas
|
||||
"neutral": "neutral",
|
||||
# Deerling / Sawsbuck
|
||||
"spring": "spring",
|
||||
"summer": "summer",
|
||||
"autumn": "autumn",
|
||||
"winter": "winter",
|
||||
# Spiky-ears Pichu
|
||||
"spiky-ears": "spiky-eared",
|
||||
# Paldean breeds
|
||||
"paldean-combat-breed": "paldea-combat-breed",
|
||||
"paldean-blaze-breed": "paldea-blaze-breed",
|
||||
"paldean-aqua-breed": "paldea-aqua-breed",
|
||||
}
|
||||
|
||||
|
||||
# PokeDB type IDs → type names (from PokeDB's type system)
|
||||
TYPE_ID_MAP: dict[int, str] = {
|
||||
1: "normal",
|
||||
2: "fighting",
|
||||
3: "flying",
|
||||
4: "poison",
|
||||
5: "ground",
|
||||
6: "rock",
|
||||
7: "bug",
|
||||
8: "ghost",
|
||||
9: "steel",
|
||||
10: "fire",
|
||||
11: "water",
|
||||
12: "grass",
|
||||
13: "electric",
|
||||
14: "psychic",
|
||||
15: "ice",
|
||||
16: "dragon",
|
||||
17: "dark",
|
||||
18: "fairy",
|
||||
}
|
||||
|
||||
|
||||
def _normalize_slug(identifier: str) -> str:
|
||||
"""Normalize a PokeDB pokemon_form_identifier to a PokeAPI-style slug.
|
||||
|
||||
PokeDB uses "pidgey-default" for base forms — strip the "-default" suffix.
|
||||
For alternate forms, translate PokeDB naming conventions to ours.
|
||||
"""
|
||||
if identifier.endswith("-default"):
|
||||
return identifier[: -len("-default")]
|
||||
|
||||
# Try suffix-based mapping: split into species + form suffix
|
||||
# e.g. "rattata-alolan" → species="rattata", suffix="alolan"
|
||||
# e.g. "mr-mime-galarian" → need to find the right split point
|
||||
# Strategy: try longest suffix first
|
||||
for pokedb_suffix, our_suffix in sorted(
|
||||
_FORM_SUFFIX_MAP.items(), key=lambda x: -len(x[0])
|
||||
):
|
||||
if identifier.endswith("-" + pokedb_suffix):
|
||||
species = identifier[: -(len(pokedb_suffix) + 1)]
|
||||
return f"{species}-{our_suffix}"
|
||||
|
||||
return identifier
|
||||
|
||||
|
||||
def _name_to_slug(name: str) -> str:
|
||||
"""Convert a display name to a PokeAPI-style slug.
|
||||
|
||||
"Bulbasaur" → "bulbasaur"
|
||||
"Mr. Mime" → "mr-mime"
|
||||
"Farfetch'd" → "farfetchd"
|
||||
"Nidoran♀" → "nidoran-f"
|
||||
"Nidoran♂" → "nidoran-m"
|
||||
"Flabébé" → "flabebe"
|
||||
"Type: Null" → "type-null"
|
||||
"""
|
||||
s = name.lower()
|
||||
s = s.replace("♀", "-f").replace("♂", "-m")
|
||||
s = s.replace("'", "").replace("'", "").replace(".", "").replace(":", "")
|
||||
s = s.replace("é", "e").replace("É", "e")
|
||||
s = s.replace(" ", "-")
|
||||
# Collapse multiple hyphens
|
||||
s = re.sub(r"-+", "-", s)
|
||||
return s.strip("-")
|
||||
|
||||
|
||||
def _name_to_form_slug(name: str) -> str | None:
|
||||
"""Convert a display name with form suffix to a PokeAPI-style slug.
|
||||
|
||||
"Rattata (Alola)" → "rattata-alola"
|
||||
"Basculin (Blue Striped)" → "basculin-blue-striped"
|
||||
"Deoxys Normal" → "deoxys-normal" (space-separated variant)
|
||||
"""
|
||||
# Try parenthesized form: "Base (Suffix)"
|
||||
m = re.match(r"^(.+?)\s*\((.+)\)$", name)
|
||||
if m:
|
||||
base = _name_to_slug(m.group(1))
|
||||
suffix = _name_to_slug(m.group(2))
|
||||
return f"{base}-{suffix}"
|
||||
|
||||
# Try space-separated form: "Deoxys Normal"
|
||||
parts = name.split()
|
||||
if len(parts) >= 2:
|
||||
return _name_to_slug(name)
|
||||
|
||||
return None
|
||||
|
||||
|
||||
# Manual overrides for PokeDB identifiers that can't be resolved generically.
|
||||
# These are cases where our pokemon.json uses non-standard base form names
|
||||
# (e.g. "Deoxys Normal" instead of "Deoxys").
|
||||
_FORM_OVERRIDES: dict[str, tuple[int, str]] = {
|
||||
"deoxys-default": (386, "Deoxys Normal"),
|
||||
"darmanitan-galarian": (10177, "Darmanitan (Galar Standard)"),
|
||||
"mimikyu-totem": (10144, "Mimikyu (Totem Disguised)"),
|
||||
"squawkabilly-green": (931, "Squawkabilly Green Plumage"),
|
||||
"squawkabilly-blue": (10260, "Squawkabilly (Blue Plumage)"),
|
||||
"squawkabilly-white": (10262, "Squawkabilly (White Plumage)"),
|
||||
"squawkabilly-yellow": (10261, "Squawkabilly (Yellow Plumage)"),
|
||||
"toxtricity-gigantamax": (849, "Toxtricity Amped"),
|
||||
}
|
||||
|
||||
|
||||
class PokemonMapper:
|
||||
"""Maps PokeDB pokemon_form_identifier → (pokeapi_id, display_name)."""
|
||||
|
||||
def __init__(self, pokemon_json_path: Path, pokedb: PokeDBData) -> None:
|
||||
# Build slug → (pokeapi_id, name) from existing pokemon.json
|
||||
self._slug_to_info: dict[str, tuple[int, str]] = {}
|
||||
self._id_to_info: dict[int, tuple[int, str]] = {} # pokeapi_id → (national_dex, name)
|
||||
self._existing_types: dict[int, list[str]] = {} # pokeapi_id → types (fallback)
|
||||
self._unmapped: set[str] = set()
|
||||
|
||||
if pokemon_json_path.exists():
|
||||
with open(pokemon_json_path) as f:
|
||||
pokemon_list = json.load(f)
|
||||
|
||||
for p in pokemon_list:
|
||||
pid = p["pokeapi_id"]
|
||||
name = p["name"]
|
||||
ndex = p["national_dex"]
|
||||
self._id_to_info[pid] = (ndex, name)
|
||||
if p.get("types"):
|
||||
self._existing_types[pid] = p["types"]
|
||||
|
||||
# Index by base slug (from pokeapi_id for base forms)
|
||||
slug = _name_to_slug(name)
|
||||
self._slug_to_info[slug] = (pid, name)
|
||||
|
||||
# Also index by form slug if it has a form suffix
|
||||
form_slug = _name_to_form_slug(name)
|
||||
if form_slug and form_slug != slug:
|
||||
self._slug_to_info[form_slug] = (pid, name)
|
||||
|
||||
# Build index from PokeDB pokemon_forms.json if it has useful fields
|
||||
self._pokedb_form_index: dict[str, dict] = {}
|
||||
# Reverse index: pokeapi_id → PokeDB form record (for non-encountered lookups)
|
||||
self._id_to_pokedb_form: dict[int, dict] = {}
|
||||
for form in pokedb.pokemon_forms:
|
||||
identifier = form.get("identifier", "")
|
||||
if identifier:
|
||||
self._pokedb_form_index[identifier] = form
|
||||
|
||||
# Build reverse index from pokeapi_id → PokeDB form
|
||||
# First, for all encountered lookups that succeed, we cache the mapping.
|
||||
# Here we pre-build for default forms using ndex_id.
|
||||
for form in pokedb.pokemon_forms:
|
||||
ndex = form.get("ndex_id")
|
||||
if ndex and form.get("is_default_form"):
|
||||
# Default form matches the base species (ndex == pokeapi_id for base forms)
|
||||
if ndex in self._id_to_info:
|
||||
self._id_to_pokedb_form[ndex] = form
|
||||
# Also look for alternate-form pokeapi_ids that share the same ndex
|
||||
for pid, (p_ndex, _) in self._id_to_info.items():
|
||||
if p_ndex == ndex and pid not in self._id_to_pokedb_form:
|
||||
self._id_to_pokedb_form[pid] = form
|
||||
|
||||
# Map non-default forms to their specific pokeapi_ids where possible
|
||||
for form in pokedb.pokemon_forms:
|
||||
identifier = form.get("identifier", "")
|
||||
if not identifier or form.get("is_default_form"):
|
||||
continue
|
||||
slug = _normalize_slug(identifier)
|
||||
if slug in self._slug_to_info:
|
||||
pid, _ = self._slug_to_info[slug]
|
||||
self._id_to_pokedb_form[pid] = form
|
||||
|
||||
def lookup(self, pokemon_form_identifier: str | None) -> tuple[int, str] | None:
|
||||
"""Look up a PokeDB pokemon_form_identifier.
|
||||
|
||||
Returns (pokeapi_id, display_name) or None if unmapped.
|
||||
"""
|
||||
if not pokemon_form_identifier:
|
||||
return None
|
||||
|
||||
# Check manual overrides first
|
||||
if pokemon_form_identifier in _FORM_OVERRIDES:
|
||||
return _FORM_OVERRIDES[pokemon_form_identifier]
|
||||
|
||||
slug = _normalize_slug(pokemon_form_identifier)
|
||||
|
||||
# Direct slug match
|
||||
if slug in self._slug_to_info:
|
||||
return self._slug_to_info[slug]
|
||||
|
||||
# Try the PokeDB form record for a pokemon_id field
|
||||
form_record = self._pokedb_form_index.get(pokemon_form_identifier, {})
|
||||
pokemon_id = form_record.get("pokemon_id")
|
||||
if pokemon_id and pokemon_id in self._id_to_info:
|
||||
ndex, name = self._id_to_info[pokemon_id]
|
||||
# Cache for future lookups
|
||||
self._slug_to_info[slug] = (pokemon_id, name)
|
||||
return (pokemon_id, name)
|
||||
|
||||
# Fallback: strip form suffix to find base species.
|
||||
# Many cosmetic forms (colors, genders, seasons) don't have separate
|
||||
# entries in our pokemon.json — they use the base species entry.
|
||||
# Try progressively shorter slugs: "flabebe-blue" → "flabebe"
|
||||
parts = slug.split("-")
|
||||
for i in range(len(parts) - 1, 0, -1):
|
||||
base = "-".join(parts[:i])
|
||||
if base in self._slug_to_info:
|
||||
result = self._slug_to_info[base]
|
||||
# Cache for future lookups
|
||||
self._slug_to_info[slug] = result
|
||||
return result
|
||||
|
||||
# Track unmapped
|
||||
if pokemon_form_identifier not in self._unmapped:
|
||||
self._unmapped.add(pokemon_form_identifier)
|
||||
|
||||
return None
|
||||
|
||||
def lookup_by_id(self, pokeapi_id: int) -> tuple[int, str] | None:
|
||||
"""Look up a pokemon by its pokeapi_id.
|
||||
|
||||
Returns (pokeapi_id, display_name) or None if not found.
|
||||
"""
|
||||
if pokeapi_id in self._id_to_info:
|
||||
_, name = self._id_to_info[pokeapi_id]
|
||||
return (pokeapi_id, name)
|
||||
return None
|
||||
|
||||
def get_sprite_url(self, pokemon_form_identifier: str | None) -> str | None:
|
||||
"""Get the PokeDB CDN sprite URL (100x100 medium) for a form identifier."""
|
||||
if not pokemon_form_identifier:
|
||||
return None
|
||||
form_record = self._pokedb_form_index.get(pokemon_form_identifier, {})
|
||||
return form_record.get("main_image_normal_path_medium")
|
||||
|
||||
def get_form_data(self, pokemon_form_identifier: str | None) -> dict | None:
|
||||
"""Get the full PokeDB form record for a form identifier."""
|
||||
if not pokemon_form_identifier:
|
||||
return None
|
||||
return self._pokedb_form_index.get(pokemon_form_identifier)
|
||||
|
||||
def all_pokemon(self) -> list[tuple[int, tuple[int, str]]]:
|
||||
"""Return all known pokemon as [(pokeapi_id, (national_dex, name)), ...].
|
||||
|
||||
Sourced from the existing pokemon.json.
|
||||
"""
|
||||
return sorted(self._id_to_info.items())
|
||||
|
||||
def get_types_for_id(self, pokeapi_id: int) -> list[str]:
|
||||
"""Get types for a pokemon by pokeapi_id, looking up via PokeDB form data.
|
||||
|
||||
Falls back to existing types from pokemon.json if no PokeDB form found.
|
||||
"""
|
||||
form = self._find_form_for_id(pokeapi_id)
|
||||
if form:
|
||||
types = []
|
||||
t1 = form.get("type_1_id")
|
||||
t2 = form.get("type_2_id")
|
||||
if t1 and t1 in TYPE_ID_MAP:
|
||||
types.append(TYPE_ID_MAP[t1])
|
||||
if t2 and t2 in TYPE_ID_MAP:
|
||||
types.append(TYPE_ID_MAP[t2])
|
||||
if types:
|
||||
return types
|
||||
# Fallback to existing types from pokemon.json
|
||||
return self._existing_types.get(pokeapi_id, [])
|
||||
|
||||
def _find_form_for_id(self, pokeapi_id: int) -> dict | None:
|
||||
"""Find the PokeDB form record for a pokeapi_id."""
|
||||
# Check pre-built reverse index first
|
||||
if pokeapi_id in self._id_to_pokedb_form:
|
||||
return self._id_to_pokedb_form[pokeapi_id]
|
||||
|
||||
if pokeapi_id not in self._id_to_info:
|
||||
return None
|
||||
_, name = self._id_to_info[pokeapi_id]
|
||||
slug = _name_to_slug(name)
|
||||
for suffix in ["-default", ""]:
|
||||
candidate = slug + suffix
|
||||
if candidate in self._pokedb_form_index:
|
||||
return self._pokedb_form_index[candidate]
|
||||
form_slug = _name_to_form_slug(name)
|
||||
if form_slug:
|
||||
for suffix in ["-default", ""]:
|
||||
candidate = form_slug + suffix
|
||||
if candidate in self._pokedb_form_index:
|
||||
return self._pokedb_form_index[candidate]
|
||||
return None
|
||||
|
||||
def has_sprite_for_id(self, pokeapi_id: int) -> bool:
|
||||
"""Check if a sprite exists for a pokemon by pokeapi_id."""
|
||||
form = self._find_form_for_id(pokeapi_id)
|
||||
return bool(form and form.get("main_image_normal_path_medium"))
|
||||
|
||||
def get_sprite_url_for_id(self, pokeapi_id: int) -> str | None:
|
||||
"""Get the PokeDB CDN sprite URL for a pokemon by pokeapi_id."""
|
||||
form = self._find_form_for_id(pokeapi_id)
|
||||
return form.get("main_image_normal_path_medium") if form else None
|
||||
|
||||
def report_unmapped(self) -> None:
|
||||
"""Print warnings for any unmapped identifiers."""
|
||||
if self._unmapped:
|
||||
print(
|
||||
f"\nWarning: {len(self._unmapped)} unmapped pokemon form identifiers:",
|
||||
file=sys.stderr,
|
||||
)
|
||||
for ident in sorted(self._unmapped):
|
||||
print(f" - {ident}", file=sys.stderr)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Location area mapping
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Region prefixes to strip from location identifiers (matching Go tool behavior).
|
||||
_REGION_PREFIXES = [
|
||||
"kanto-", "johto-", "hoenn-", "sinnoh-",
|
||||
"unova-", "kalos-", "alola-", "galar-",
|
||||
"hisui-", "paldea-",
|
||||
]
|
||||
|
||||
|
||||
def _identifier_to_name(identifier: str) -> str:
|
||||
"""Convert a hyphenated identifier to a Title Case display name.
|
||||
|
||||
"route-01-kanto" → "Route 01 Kanto" (region stripping done separately)
|
||||
"viridian-forest" → "Viridian Forest"
|
||||
"""
|
||||
return identifier.replace("-", " ").title()
|
||||
|
||||
|
||||
class LocationMapper:
|
||||
"""Maps PokeDB location_area_identifier → (location_name, area_suffix)."""
|
||||
|
||||
def __init__(self, pokedb: PokeDBData) -> None:
|
||||
# Build location_area_identifier → location_identifier lookup
|
||||
self._area_to_location: dict[str, str] = {}
|
||||
# location_identifier → location display name
|
||||
self._location_names: dict[str, str] = {}
|
||||
# location_area_identifier → area display name
|
||||
self._area_names: dict[str, str] = {}
|
||||
|
||||
# Index locations
|
||||
for loc in pokedb.locations:
|
||||
identifier = loc.get("identifier", "")
|
||||
name = loc.get("name", "")
|
||||
if identifier:
|
||||
self._location_names[identifier] = name if name else self._clean_location_name(identifier)
|
||||
|
||||
# Index location areas
|
||||
for area in pokedb.location_areas:
|
||||
area_id = area.get("identifier", "")
|
||||
loc_id = area.get("location_identifier", "")
|
||||
area_name = area.get("name", "")
|
||||
if area_id:
|
||||
self._area_to_location[area_id] = loc_id
|
||||
self._area_names[area_id] = area_name if area_name else ""
|
||||
|
||||
@staticmethod
|
||||
def _clean_location_name(identifier: str) -> str:
|
||||
"""Clean a location identifier into a display name."""
|
||||
name = identifier
|
||||
for prefix in _REGION_PREFIXES:
|
||||
if name.startswith(prefix):
|
||||
name = name[len(prefix):]
|
||||
break
|
||||
return _identifier_to_name(name)
|
||||
|
||||
def get_location_name(self, location_area_identifier: str) -> str:
|
||||
"""Get the display name for a location area's parent location."""
|
||||
loc_id = self._area_to_location.get(location_area_identifier, "")
|
||||
if loc_id and loc_id in self._location_names:
|
||||
return self._location_names[loc_id]
|
||||
# Fallback: derive from the area identifier itself
|
||||
return self._clean_location_name(location_area_identifier)
|
||||
|
||||
def get_area_name(self, location_area_identifier: str) -> str:
|
||||
"""Get the display name for a specific location area."""
|
||||
return self._area_names.get(location_area_identifier, "")
|
||||
|
||||
def get_location_identifier(self, location_area_identifier: str) -> str:
|
||||
"""Get the parent location identifier for a location area."""
|
||||
return self._area_to_location.get(location_area_identifier, "")
|
||||
@@ -0,0 +1,81 @@
|
||||
"""Output data models matching the existing seed JSON format."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass, field
|
||||
|
||||
|
||||
@dataclass
|
||||
class Encounter:
|
||||
pokeapi_id: int
|
||||
pokemon_name: str
|
||||
method: str
|
||||
encounter_rate: int
|
||||
min_level: int
|
||||
max_level: int
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"pokeapi_id": self.pokeapi_id,
|
||||
"pokemon_name": self.pokemon_name,
|
||||
"method": self.method,
|
||||
"encounter_rate": self.encounter_rate,
|
||||
"min_level": self.min_level,
|
||||
"max_level": self.max_level,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Route:
|
||||
name: str
|
||||
order: int
|
||||
encounters: list[Encounter] = field(default_factory=list)
|
||||
children: list[Route] = field(default_factory=list)
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
d: dict = {
|
||||
"name": self.name,
|
||||
"order": self.order,
|
||||
"encounters": [e.to_dict() for e in self.encounters],
|
||||
}
|
||||
if self.children:
|
||||
d["children"] = [c.to_dict() for c in self.children]
|
||||
return d
|
||||
|
||||
|
||||
@dataclass
|
||||
class Game:
|
||||
name: str
|
||||
slug: str
|
||||
generation: int
|
||||
region: str
|
||||
release_year: int
|
||||
color: str | None = None
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"name": self.name,
|
||||
"slug": self.slug,
|
||||
"generation": self.generation,
|
||||
"region": self.region,
|
||||
"release_year": self.release_year,
|
||||
"color": self.color,
|
||||
}
|
||||
|
||||
|
||||
@dataclass
|
||||
class Pokemon:
|
||||
pokeapi_id: int
|
||||
national_dex: int
|
||||
name: str
|
||||
types: list[str]
|
||||
sprite_url: str
|
||||
|
||||
def to_dict(self) -> dict:
|
||||
return {
|
||||
"pokeapi_id": self.pokeapi_id,
|
||||
"national_dex": self.national_dex,
|
||||
"name": self.name,
|
||||
"types": self.types,
|
||||
"sprite_url": self.sprite_url,
|
||||
}
|
||||
@@ -0,0 +1,289 @@
|
||||
"""Output seed JSON files in the existing format."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import json
|
||||
import re
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .loader import SeedConfig
|
||||
from .mappings import TYPE_ID_MAP, PokemonMapper
|
||||
from .models import Encounter, Route
|
||||
from .sprites import sprite_path_for_pokemon
|
||||
|
||||
# Max route name length (matches DB column VARCHAR(100))
|
||||
_MAX_ROUTE_NAME_LEN = 100
|
||||
|
||||
# Pattern for Sw/Sh den area names: "Location (Den X - long description - Common/Rare)"
|
||||
_DEN_NAME_RE = re.compile(r"^(.+?) \(Den ([A-Z]\d*) - .+ - (Common|Rare)\)$")
|
||||
|
||||
|
||||
def _truncate_route_name(name: str) -> str:
|
||||
"""Truncate a route name to fit the database column limit.
|
||||
|
||||
Applies smart truncation for known patterns (e.g. Sw/Sh den descriptions).
|
||||
"""
|
||||
if len(name) <= _MAX_ROUTE_NAME_LEN:
|
||||
return name
|
||||
|
||||
# Sw/Sh dens: shorten "Location (Den X - long desc - Common/Rare)" → "Location (Den X - Common/Rare)"
|
||||
m = _DEN_NAME_RE.match(name)
|
||||
if m:
|
||||
shortened = f"{m.group(1)} (Den {m.group(2)} - {m.group(3)})"
|
||||
if len(shortened) <= _MAX_ROUTE_NAME_LEN:
|
||||
return shortened
|
||||
|
||||
# Generic truncation: cut at last space before limit, add ellipsis
|
||||
truncated = name[:_MAX_ROUTE_NAME_LEN - 1].rsplit(" ", 1)[0] + "…"
|
||||
return truncated
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Route ordering
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _get_route_order(config: SeedConfig, vg_key: str) -> list[str] | None:
|
||||
"""Get the route order list for a version group, resolving aliases."""
|
||||
return config.route_order.get(vg_key)
|
||||
|
||||
|
||||
def _route_sort_key(name: str, order_list: list[str]) -> tuple[int, str]:
|
||||
"""Compute sort key for a route name against an ordered list."""
|
||||
for i, ordered_name in enumerate(order_list):
|
||||
if name == ordered_name or name.startswith(ordered_name + " ("):
|
||||
return (i, name)
|
||||
return (len(order_list), name)
|
||||
|
||||
|
||||
def sort_routes(routes: list[Route], config: SeedConfig, vg_key: str) -> list[Route]:
|
||||
"""Sort routes by game progression order and assign sequential order values."""
|
||||
order_list = _get_route_order(config, vg_key)
|
||||
if order_list:
|
||||
routes.sort(key=lambda r: _route_sort_key(r.name, order_list))
|
||||
else:
|
||||
print(f" Warning: No route order for {vg_key}, using default order", file=sys.stderr)
|
||||
|
||||
# Assign sequential order values
|
||||
order = 1
|
||||
for route in routes:
|
||||
route.order = order
|
||||
order += 1
|
||||
for child in route.children:
|
||||
child.order = order
|
||||
order += 1
|
||||
|
||||
return routes
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Special encounters
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def _get_special_encounters(
|
||||
config: SeedConfig,
|
||||
vg_key: str,
|
||||
) -> dict[str, list[dict[str, Any]]] | None:
|
||||
"""Get special encounters for a version group, resolving aliases."""
|
||||
if not config.special_encounters:
|
||||
return None
|
||||
|
||||
encounters = config.special_encounters.get("encounters", {})
|
||||
aliases = config.special_encounters.get("aliases", {})
|
||||
|
||||
if vg_key in encounters:
|
||||
return encounters[vg_key]
|
||||
if vg_key in aliases:
|
||||
alias_target = aliases[vg_key]
|
||||
if alias_target in encounters:
|
||||
return encounters[alias_target]
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def merge_special_encounters(
|
||||
routes: list[Route],
|
||||
config: SeedConfig,
|
||||
vg_key: str,
|
||||
pokemon_mapper: PokemonMapper | None = None,
|
||||
) -> list[Route]:
|
||||
"""Merge special encounters (starters, gifts, fossils) into routes."""
|
||||
special_data = _get_special_encounters(config, vg_key)
|
||||
if not special_data:
|
||||
return routes
|
||||
|
||||
# Build lookup: route name → route (including children)
|
||||
route_map: dict[str, Route] = {}
|
||||
for route in routes:
|
||||
route_map[route.name] = route
|
||||
for child in route.children:
|
||||
route_map[child.name] = child
|
||||
|
||||
for location_name, encounter_dicts in special_data.items():
|
||||
encounters = []
|
||||
for e in encounter_dicts:
|
||||
# Look up proper display name from pokemon mapper if available
|
||||
pokemon_name = e["pokemon_name"]
|
||||
if pokemon_mapper:
|
||||
info = pokemon_mapper.lookup_by_id(e["pokeapi_id"])
|
||||
if info:
|
||||
pokemon_name = info[1]
|
||||
|
||||
encounters.append(Encounter(
|
||||
pokeapi_id=e["pokeapi_id"],
|
||||
pokemon_name=pokemon_name,
|
||||
method=e["method"],
|
||||
encounter_rate=e["encounter_rate"],
|
||||
min_level=e["min_level"],
|
||||
max_level=e["max_level"],
|
||||
))
|
||||
|
||||
if location_name in route_map:
|
||||
route_map[location_name].encounters.extend(encounters)
|
||||
else:
|
||||
new_route = Route(name=location_name, order=0, encounters=encounters)
|
||||
routes.append(new_route)
|
||||
route_map[location_name] = new_route
|
||||
|
||||
return routes
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# JSON output
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def write_json(path: Path, data: Any) -> None:
|
||||
"""Write data as formatted JSON with trailing newline."""
|
||||
content = json.dumps(data, indent=2, ensure_ascii=False)
|
||||
path.write_text(content + "\n", encoding="utf-8")
|
||||
print(f" -> {path}")
|
||||
|
||||
|
||||
def _deduplicate_names(routes: list[Route]) -> None:
|
||||
"""Ensure all route names are unique by appending a numeric suffix to duplicates."""
|
||||
seen: dict[str, int] = {}
|
||||
|
||||
def _unique(name: str) -> str:
|
||||
if name not in seen:
|
||||
seen[name] = 1
|
||||
return name
|
||||
seen[name] += 1
|
||||
return f"{name} #{seen[name]}"
|
||||
|
||||
for route in routes:
|
||||
route.name = _unique(_truncate_route_name(route.name))
|
||||
for child in route.children:
|
||||
child.name = _unique(_truncate_route_name(child.name))
|
||||
|
||||
|
||||
def write_game_json(routes: list[Route], output_dir: Path, game_slug: str) -> None:
|
||||
"""Write a per-game route/encounter JSON file."""
|
||||
_deduplicate_names(routes)
|
||||
data = [r.to_dict() for r in routes]
|
||||
write_json(output_dir / f"{game_slug}.json", data)
|
||||
|
||||
|
||||
_GAME_CATEGORY: dict[str, str] = {
|
||||
"red": "original", "blue": "original", "yellow": "enhanced",
|
||||
"gold": "original", "silver": "original", "crystal": "enhanced",
|
||||
"ruby": "original", "sapphire": "original", "emerald": "enhanced",
|
||||
"firered": "remake", "leafgreen": "remake",
|
||||
"diamond": "original", "pearl": "original", "platinum": "enhanced",
|
||||
"heartgold": "remake", "soulsilver": "remake",
|
||||
"black": "original", "white": "original",
|
||||
"black-2": "sequel", "white-2": "sequel",
|
||||
"x": "original", "y": "original",
|
||||
"omega-ruby": "remake", "alpha-sapphire": "remake",
|
||||
"sun": "original", "moon": "original",
|
||||
"ultra-sun": "enhanced", "ultra-moon": "enhanced",
|
||||
"lets-go-pikachu": "remake", "lets-go-eevee": "remake",
|
||||
"sword": "original", "shield": "original",
|
||||
"brilliant-diamond": "remake", "shining-pearl": "remake",
|
||||
"legends-arceus": "spinoff",
|
||||
"scarlet": "original", "violet": "original",
|
||||
"legends-z-a": "spinoff",
|
||||
}
|
||||
|
||||
|
||||
def write_games_json(config: SeedConfig, output_dir: Path) -> None:
|
||||
"""Write games.json from version_groups config."""
|
||||
games = []
|
||||
for vg_info in config.version_groups.values():
|
||||
for game_info in vg_info.get("games", {}).values():
|
||||
slug = game_info["slug"]
|
||||
games.append({
|
||||
"name": game_info["name"],
|
||||
"slug": slug,
|
||||
"generation": vg_info["generation"],
|
||||
"region": vg_info["region"],
|
||||
"release_year": game_info["release_year"],
|
||||
"color": game_info.get("color"),
|
||||
"category": _GAME_CATEGORY.get(slug, "original"),
|
||||
})
|
||||
write_json(output_dir / "games.json", games)
|
||||
print(f" Wrote {len(games)} games")
|
||||
|
||||
|
||||
def write_pokemon_json(
|
||||
pokemon_mapper: PokemonMapper,
|
||||
encountered_form_ids: set[str],
|
||||
sprite_map: dict[str, str],
|
||||
output_dir: Path,
|
||||
) -> None:
|
||||
"""Write pokemon.json with all known pokemon.
|
||||
|
||||
Includes all pokemon from the existing pokemon.json (base data),
|
||||
enriched with PokeDB types and sprite paths for encountered forms.
|
||||
"""
|
||||
# Build a mapping of pokeapi_id → (form_id, form_data) for encountered forms
|
||||
encountered_by_id: dict[int, tuple[str, dict[str, Any] | None]] = {}
|
||||
for form_id in encountered_form_ids:
|
||||
info = pokemon_mapper.lookup(form_id)
|
||||
if info is None:
|
||||
continue
|
||||
pokeapi_id, _ = info
|
||||
if pokeapi_id not in encountered_by_id:
|
||||
form_data = pokemon_mapper.get_form_data(form_id)
|
||||
encountered_by_id[pokeapi_id] = (form_id, form_data)
|
||||
|
||||
pokemon_list: list[dict[str, Any]] = []
|
||||
|
||||
for pokeapi_id, (ndex, name) in pokemon_mapper.all_pokemon():
|
||||
# Enrich with PokeDB data if this pokemon was encountered
|
||||
if pokeapi_id in encountered_by_id:
|
||||
form_id, form_data = encountered_by_id[pokeapi_id]
|
||||
types = _extract_types(form_data) if form_data else []
|
||||
sprite_url = sprite_path_for_pokemon(pokeapi_id) if form_id in sprite_map else None
|
||||
national_dex = form_data.get("ndex_id", ndex) if form_data else ndex
|
||||
else:
|
||||
# Not encountered — use existing data, try to find PokeDB form for types
|
||||
types = pokemon_mapper.get_types_for_id(pokeapi_id)
|
||||
sprite_url = sprite_path_for_pokemon(pokeapi_id) if pokemon_mapper.has_sprite_for_id(pokeapi_id) else None
|
||||
national_dex = ndex
|
||||
|
||||
pokemon_list.append({
|
||||
"pokeapi_id": pokeapi_id,
|
||||
"national_dex": national_dex,
|
||||
"name": name,
|
||||
"types": types,
|
||||
"sprite_url": sprite_url,
|
||||
})
|
||||
|
||||
# Sort by pokeapi_id
|
||||
pokemon_list.sort(key=lambda p: p["pokeapi_id"])
|
||||
|
||||
write_json(output_dir / "pokemon.json", pokemon_list)
|
||||
print(f" Wrote {len(pokemon_list)} pokemon")
|
||||
|
||||
|
||||
def _extract_types(form_data: dict[str, Any]) -> list[str]:
|
||||
"""Extract type names from a PokeDB form record."""
|
||||
types = []
|
||||
type1_id = form_data.get("type_1_id")
|
||||
type2_id = form_data.get("type_2_id")
|
||||
if type1_id and type1_id in TYPE_ID_MAP:
|
||||
types.append(TYPE_ID_MAP[type1_id])
|
||||
if type2_id and type2_id in TYPE_ID_MAP:
|
||||
types.append(TYPE_ID_MAP[type2_id])
|
||||
return types
|
||||
@@ -0,0 +1,343 @@
|
||||
"""Core encounter processing: filter, parse, aggregate, and group encounters."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import re
|
||||
from typing import Any
|
||||
|
||||
from .mappings import LocationMapper, PokemonMapper, map_encounter_method
|
||||
from .models import Encounter, Route
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Rate parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
# Word-based rates → numeric value
|
||||
_WORD_RATES: dict[str, int] = {
|
||||
"one": 100,
|
||||
"two": 50,
|
||||
"three": 33,
|
||||
"four": 25,
|
||||
"five": 20,
|
||||
"six": 17,
|
||||
"seven": 14,
|
||||
"eight": 13,
|
||||
"choose one": 100,
|
||||
"one of three": 33,
|
||||
"only one": 100,
|
||||
"unlimited": 100,
|
||||
"respawns": 100,
|
||||
"common": 60,
|
||||
"average": 30,
|
||||
"rare": 10,
|
||||
"varies": 50,
|
||||
}
|
||||
|
||||
_PERCENT_RE = re.compile(r"~?(\d+(?:\.\d+)?)%?")
|
||||
|
||||
|
||||
def parse_rate(value: str | None) -> int | None:
|
||||
"""Parse a rate string into an integer percentage (0-100).
|
||||
|
||||
Handles formats: "50%", "~10%", "one", "common", "100", "??%", etc.
|
||||
Returns None if unparseable.
|
||||
"""
|
||||
if not value:
|
||||
return None
|
||||
|
||||
value = value.strip()
|
||||
|
||||
# Word-based
|
||||
lower = value.lower()
|
||||
if lower in _WORD_RATES:
|
||||
return _WORD_RATES[lower]
|
||||
|
||||
# Unknown
|
||||
if value == "??%":
|
||||
return None
|
||||
|
||||
# Numeric percentage: "50%", "~10%", "10.14%", or bare "100"
|
||||
m = _PERCENT_RE.match(value)
|
||||
if m:
|
||||
return max(1, round(float(m.group(1))))
|
||||
|
||||
return None
|
||||
|
||||
|
||||
def extract_encounter_rate(record: dict[str, Any], generation: int) -> int:
|
||||
"""Extract a single encounter_rate from a PokeDB encounter record.
|
||||
|
||||
Flattens generation-specific rate variants into a single value.
|
||||
"""
|
||||
# Gen 1/3/6: rate_overall
|
||||
rate_overall = parse_rate(record.get("rate_overall"))
|
||||
if rate_overall is not None:
|
||||
return rate_overall
|
||||
|
||||
# Gen 2/4: time-of-day rates — take the max
|
||||
time_rates = [
|
||||
parse_rate(record.get("rate_morning")),
|
||||
parse_rate(record.get("rate_day")),
|
||||
parse_rate(record.get("rate_night")),
|
||||
]
|
||||
time_rates = [r for r in time_rates if r is not None]
|
||||
if time_rates:
|
||||
return max(time_rates)
|
||||
|
||||
# Gen 5: seasonal rates — take the max
|
||||
season_rates = [
|
||||
parse_rate(record.get("rate_spring")),
|
||||
parse_rate(record.get("rate_summer")),
|
||||
parse_rate(record.get("rate_autumn")),
|
||||
parse_rate(record.get("rate_winter")),
|
||||
]
|
||||
season_rates = [r for r in season_rates if r is not None]
|
||||
if season_rates:
|
||||
return max(season_rates)
|
||||
|
||||
# Gen 8 Sw/Sh: weather rates — take the max
|
||||
weather_rates = []
|
||||
for key, val in record.items():
|
||||
if key.startswith("weather_") and key.endswith("_rate") and val:
|
||||
parsed = parse_rate(val)
|
||||
if parsed is not None:
|
||||
weather_rates.append(parsed)
|
||||
if weather_rates:
|
||||
return max(weather_rates)
|
||||
|
||||
# Gen 8 Legends Arceus: boolean conditions → presence-based
|
||||
if record.get("during_any_time") or record.get("during_morning") or \
|
||||
record.get("during_day") or record.get("during_evening") or record.get("during_night"):
|
||||
return 100 # Present under conditions
|
||||
|
||||
# Gen 9 Sc/Vi: probability weights → normalize
|
||||
prob_overall = record.get("probability_overall")
|
||||
if prob_overall:
|
||||
parsed = parse_rate(prob_overall)
|
||||
if parsed is not None:
|
||||
# These are spawn weights (e.g. "20", "300"), not percentages.
|
||||
# We'll normalize them later during aggregation when we have
|
||||
# all encounters for a location. For now, store the raw weight.
|
||||
return parsed
|
||||
|
||||
# Check time-based probability variants
|
||||
prob_rates = [
|
||||
parse_rate(record.get("probability_morning")),
|
||||
parse_rate(record.get("probability_day")),
|
||||
parse_rate(record.get("probability_evening")),
|
||||
parse_rate(record.get("probability_night")),
|
||||
]
|
||||
prob_rates = [r for r in prob_rates if r is not None]
|
||||
if prob_rates:
|
||||
return max(prob_rates)
|
||||
|
||||
# Fallback: gift/trade/static encounters with no rate
|
||||
return 100
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Level parsing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def parse_levels(levels_str: str | None) -> tuple[int, int]:
|
||||
"""Parse a level string into (min_level, max_level).
|
||||
|
||||
"2 - 4" → (2, 4)
|
||||
"67" → (67, 67)
|
||||
"44 - 51" → (44, 51)
|
||||
Returns (1, 1) if unparseable.
|
||||
"""
|
||||
if not levels_str:
|
||||
return (1, 1)
|
||||
|
||||
levels_str = levels_str.strip()
|
||||
|
||||
# Range: "2 - 4" or "2-4"
|
||||
m = re.match(r"(\d+)\s*-\s*(\d+)", levels_str)
|
||||
if m:
|
||||
return (int(m.group(1)), int(m.group(2)))
|
||||
|
||||
# Single: "67"
|
||||
m = re.match(r"(\d+)", levels_str)
|
||||
if m:
|
||||
level = int(m.group(1))
|
||||
return (level, level)
|
||||
|
||||
return (1, 1)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# Core processing
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
def filter_encounters_for_game(
|
||||
encounters: list[dict[str, Any]],
|
||||
game_slug: str,
|
||||
) -> list[dict[str, Any]]:
|
||||
"""Filter PokeDB encounters to only those for a specific game version."""
|
||||
return [
|
||||
e for e in encounters
|
||||
if game_slug in (e.get("version_identifiers") or [])
|
||||
]
|
||||
|
||||
|
||||
def process_encounters(
|
||||
raw_encounters: list[dict[str, Any]],
|
||||
generation: int,
|
||||
pokemon_mapper: PokemonMapper,
|
||||
location_mapper: LocationMapper,
|
||||
) -> dict[str, list[Encounter]]:
|
||||
"""Process raw PokeDB encounters into grouped-by-location-area Encounter objects.
|
||||
|
||||
Returns {location_area_identifier: [Encounter, ...]}.
|
||||
"""
|
||||
by_area: dict[str, list[Encounter]] = {}
|
||||
|
||||
for record in raw_encounters:
|
||||
# Map encounter method
|
||||
method_id = record.get("encounter_method_identifier", "")
|
||||
method = map_encounter_method(method_id) if method_id else None
|
||||
if method is None:
|
||||
continue
|
||||
|
||||
# Map pokemon
|
||||
form_id = record.get("pokemon_form_identifier")
|
||||
pokemon_info = pokemon_mapper.lookup(form_id)
|
||||
if pokemon_info is None:
|
||||
continue
|
||||
|
||||
pokeapi_id, pokemon_name = pokemon_info
|
||||
|
||||
# Parse levels
|
||||
min_level, max_level = parse_levels(record.get("levels"))
|
||||
|
||||
# Extract rate
|
||||
encounter_rate = extract_encounter_rate(record, generation)
|
||||
|
||||
# Location area
|
||||
area_id = record.get("location_area_identifier", "")
|
||||
if not area_id:
|
||||
continue
|
||||
|
||||
enc = Encounter(
|
||||
pokeapi_id=pokeapi_id,
|
||||
pokemon_name=pokemon_name,
|
||||
method=method,
|
||||
encounter_rate=encounter_rate,
|
||||
min_level=min_level,
|
||||
max_level=max_level,
|
||||
)
|
||||
|
||||
by_area.setdefault(area_id, []).append(enc)
|
||||
|
||||
return by_area
|
||||
|
||||
|
||||
def aggregate_encounters(encounters: list[Encounter]) -> list[Encounter]:
|
||||
"""Aggregate encounters by (pokeapi_id, method), merging level ranges and summing rates.
|
||||
|
||||
Replicates the Go tool's aggregation logic.
|
||||
"""
|
||||
key_type = tuple[int, str]
|
||||
agg: dict[key_type, Encounter] = {}
|
||||
order: list[key_type] = []
|
||||
|
||||
for enc in encounters:
|
||||
k = (enc.pokeapi_id, enc.method)
|
||||
if k in agg:
|
||||
existing = agg[k]
|
||||
existing.encounter_rate += enc.encounter_rate
|
||||
existing.min_level = min(existing.min_level, enc.min_level)
|
||||
existing.max_level = max(existing.max_level, enc.max_level)
|
||||
else:
|
||||
# Copy so we don't mutate the original
|
||||
agg[k] = Encounter(
|
||||
pokeapi_id=enc.pokeapi_id,
|
||||
pokemon_name=enc.pokemon_name,
|
||||
method=enc.method,
|
||||
encounter_rate=enc.encounter_rate,
|
||||
min_level=enc.min_level,
|
||||
max_level=enc.max_level,
|
||||
)
|
||||
order.append(k)
|
||||
|
||||
result = []
|
||||
for k in order:
|
||||
e = agg[k]
|
||||
e.encounter_rate = min(e.encounter_rate, 100)
|
||||
result.append(e)
|
||||
|
||||
# Sort by rate descending, then name ascending
|
||||
result.sort(key=lambda e: (-e.encounter_rate, e.pokemon_name))
|
||||
return result
|
||||
|
||||
|
||||
def build_routes(
|
||||
encounters_by_area: dict[str, list[Encounter]],
|
||||
location_mapper: LocationMapper,
|
||||
) -> list[Route]:
|
||||
"""Group encounters by location, building parent/child route hierarchy.
|
||||
|
||||
Multiple areas under the same location → parent route with children.
|
||||
Single area → flat route.
|
||||
"""
|
||||
# Group areas by their parent location identifier
|
||||
loc_groups: dict[str, list[tuple[str, str, list[Encounter]]]] = {}
|
||||
# loc_id → [(area_id, area_display_name, encounters), ...]
|
||||
|
||||
for area_id, encounters in encounters_by_area.items():
|
||||
loc_id = location_mapper.get_location_identifier(area_id)
|
||||
if not loc_id:
|
||||
loc_id = area_id # fallback
|
||||
|
||||
area_name = location_mapper.get_area_name(area_id)
|
||||
loc_groups.setdefault(loc_id, []).append((area_id, area_name, encounters))
|
||||
|
||||
routes: list[Route] = []
|
||||
|
||||
for loc_id, areas in loc_groups.items():
|
||||
loc_name = location_mapper.get_location_name(areas[0][0])
|
||||
|
||||
if len(areas) == 1:
|
||||
# Single area — flat route
|
||||
_, area_name, encounters = areas[0]
|
||||
aggregated = aggregate_encounters(encounters)
|
||||
if aggregated:
|
||||
# If the area has a distinct name different from the location, use it
|
||||
route_name = area_name if area_name and area_name != loc_name else loc_name
|
||||
routes.append(Route(name=route_name, order=0, encounters=aggregated))
|
||||
|
||||
else:
|
||||
# Multiple areas — check if encounters differ
|
||||
children: list[Route] = []
|
||||
# Encounters for areas with no distinct name get merged into parent
|
||||
parent_encounters: list[Encounter] = []
|
||||
|
||||
for _, area_name, encounters in areas:
|
||||
aggregated = aggregate_encounters(encounters)
|
||||
if aggregated:
|
||||
if area_name and area_name != loc_name:
|
||||
children.append(Route(name=area_name, order=0, encounters=aggregated))
|
||||
else:
|
||||
# No distinct area name — merge into parent
|
||||
parent_encounters.extend(aggregated)
|
||||
|
||||
if children:
|
||||
# Parent with children (parent may also have its own encounters)
|
||||
parent_agg = aggregate_encounters(parent_encounters) if parent_encounters else []
|
||||
routes.append(Route(
|
||||
name=loc_name,
|
||||
order=0,
|
||||
encounters=parent_agg,
|
||||
children=children,
|
||||
))
|
||||
elif parent_encounters:
|
||||
# All areas had same name — flatten into single route
|
||||
routes.append(Route(
|
||||
name=loc_name,
|
||||
order=0,
|
||||
encounters=aggregate_encounters(parent_encounters),
|
||||
))
|
||||
|
||||
return routes
|
||||
@@ -0,0 +1,130 @@
|
||||
"""Download and manage PokeDB pokemon sprites."""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import sys
|
||||
import urllib.request
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
from .mappings import PokemonMapper
|
||||
|
||||
|
||||
def download_sprites(
|
||||
pokemon_mapper: PokemonMapper,
|
||||
encountered_form_ids: set[str],
|
||||
sprites_dir: Path,
|
||||
) -> dict[str, str]:
|
||||
"""Download sprites for all encountered pokemon forms.
|
||||
|
||||
Returns a mapping of pokemon_form_identifier → local sprite filename.
|
||||
Skips already-downloaded sprites.
|
||||
"""
|
||||
sprites_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
to_download: list[tuple[str, str, Path]] = [] # (form_id, url, dest)
|
||||
result: dict[str, str] = {}
|
||||
|
||||
for form_id in sorted(encountered_form_ids):
|
||||
info = pokemon_mapper.lookup(form_id)
|
||||
if info is None:
|
||||
continue
|
||||
|
||||
pokeapi_id, _ = info
|
||||
sprite_url = pokemon_mapper.get_sprite_url(form_id)
|
||||
if not sprite_url:
|
||||
continue
|
||||
|
||||
filename = f"{pokeapi_id}.webp"
|
||||
dest = sprites_dir / filename
|
||||
result[form_id] = filename
|
||||
|
||||
if not dest.exists():
|
||||
to_download.append((form_id, sprite_url, dest))
|
||||
|
||||
if not to_download:
|
||||
print(f" Sprites: {len(result)} already cached")
|
||||
return result
|
||||
|
||||
print(f" Downloading {len(to_download)} sprites ({len(result) - len(to_download)} cached)...")
|
||||
|
||||
failed = 0
|
||||
for i, (form_id, url, dest) in enumerate(to_download, 1):
|
||||
try:
|
||||
urllib.request.urlretrieve(url, dest)
|
||||
except Exception as e:
|
||||
print(f" Warning: Failed to download sprite for {form_id}: {e}", file=sys.stderr)
|
||||
failed += 1
|
||||
# Remove the failed entry from results
|
||||
result.pop(form_id, None)
|
||||
|
||||
# Progress every 100
|
||||
if i % 100 == 0:
|
||||
print(f" {i}/{len(to_download)}...")
|
||||
|
||||
if failed:
|
||||
print(f" Sprites: {len(result)} downloaded, {failed} failed")
|
||||
else:
|
||||
print(f" Sprites: {len(result)} total ({len(to_download)} new)")
|
||||
|
||||
return result
|
||||
|
||||
|
||||
def download_all_sprites(
|
||||
pokemon_mapper: PokemonMapper,
|
||||
sprites_dir: Path,
|
||||
) -> set[int]:
|
||||
"""Download sprites for all known pokemon (not just encountered ones).
|
||||
|
||||
Returns a set of pokeapi_ids that have sprites downloaded.
|
||||
"""
|
||||
sprites_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
to_download: list[tuple[int, str, Path]] = []
|
||||
have_sprites: set[int] = set()
|
||||
|
||||
for pokeapi_id, (_ndex, _name) in pokemon_mapper.all_pokemon():
|
||||
if pokemon_mapper.has_sprite_for_id(pokeapi_id):
|
||||
filename = f"{pokeapi_id}.webp"
|
||||
dest = sprites_dir / filename
|
||||
have_sprites.add(pokeapi_id)
|
||||
|
||||
if not dest.exists():
|
||||
# Get the sprite URL via the mapper
|
||||
url = pokemon_mapper.get_sprite_url_for_id(pokeapi_id)
|
||||
if url:
|
||||
to_download.append((pokeapi_id, url, dest))
|
||||
|
||||
if not to_download:
|
||||
print(f" All sprites: {len(have_sprites)} already cached")
|
||||
return have_sprites
|
||||
|
||||
print(f" Downloading {len(to_download)} additional sprites ({len(have_sprites) - len(to_download)} cached)...")
|
||||
|
||||
failed = 0
|
||||
for i, (pid, url, dest) in enumerate(to_download, 1):
|
||||
try:
|
||||
urllib.request.urlretrieve(url, dest)
|
||||
except Exception as e:
|
||||
print(f" Warning: Failed to download sprite for pokemon {pid}: {e}", file=sys.stderr)
|
||||
failed += 1
|
||||
have_sprites.discard(pid)
|
||||
|
||||
if i % 100 == 0:
|
||||
print(f" {i}/{len(to_download)}...")
|
||||
|
||||
if failed:
|
||||
print(f" All sprites: {len(have_sprites)} downloaded, {failed} failed")
|
||||
else:
|
||||
print(f" All sprites: {len(have_sprites)} total ({len(to_download)} new)")
|
||||
|
||||
return have_sprites
|
||||
|
||||
|
||||
def sprite_path_for_pokemon(pokeapi_id: int) -> str:
|
||||
"""Generate the sprite URL path for use in pokemon.json.
|
||||
|
||||
Returns an absolute path like "/sprites/25.webp" for the frontend
|
||||
(files in frontend/public/ are served at the root).
|
||||
"""
|
||||
return f"/sprites/{pokeapi_id}.webp"
|
||||
@@ -0,0 +1,9 @@
|
||||
[project]
|
||||
name = "import-pokedb"
|
||||
version = "0.1.0"
|
||||
description = "Convert PokeDB.org JSON data exports into nuzlocke-tracker seed format"
|
||||
requires-python = ">=3.12"
|
||||
dependencies = []
|
||||
|
||||
[project.scripts]
|
||||
import-pokedb = "import_pokedb.__main__:main"
|
||||
Reference in New Issue
Block a user