Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
60 commits
Select commit Hold shift + click to select a range
6a590d0
Add Groq LLM support, budget filtering, and TPCAgent improvements
Jun 9, 2026
aadf4b1
Add TPC IJCAI 2026 Phase 1 query files (Chinese and English)
Jun 9, 2026
c767cc1
Add LISTEN-active as submodule at listen/
Jun 9, 2026
086ab94
Add Ollama LLM support, load_dotenv in run_tpc, and pre-filter closed…
Jun 13, 2026
155bc64
Update listen submodule: fix stage2 timing-POI pin loss
Jun 14, 2026
a79e3bf
Update listen submodule: add --rerun-failures to run_tpc.py
Jun 14, 2026
82447f5
Point listen submodule to china branch (e3d8c75)
Jun 14, 2026
58008d4
Update listen submodule: add analyze_run.py TPC failure analysis tool
Jun 14, 2026
5a0c3a6
Update listen submodule: fix run_topk tuple unpacking bug in solve()
Jun 14, 2026
2b2992f
Update listen submodule: add B&B solver (bnb_agent.py, --algo bnb)
Jun 14, 2026
99c9c1a
Update listen submodule: fix .env upward search for parent-repo layout
Jun 14, 2026
12028ba
Update listen submodule: fix chinatravel root path detection
Jun 14, 2026
fe92850
Update listen submodule: fix remaining CT root path issues in candida…
Jun 14, 2026
aa638c6
Update listen submodule: fix poi_distance, apostrophe exec, intercity…
Jun 15, 2026
5aff7a0
Update listen submodule: deterministic two-phase tournament ranking +…
Jun 15, 2026
26d0762
Update listen submodule: Round 2 B&B fixes (hotel case, closest-hotel)
Jun 15, 2026
9290d04
Update listen submodule: Round 4 B&B fixes (City Park, timing, dual-t…
Jun 15, 2026
e68f1ad
Update listen submodule: add --bnb-batches flag
Jun 15, 2026
d18d26c
Update listen submodule: required attraction name injection + CLI flags
Jun 15, 2026
fc1a159
Update listen submodule: ATTRACTION_VISIT_TIME end_time floor + arriv…
Jun 15, 2026
6303f5e
Update listen submodule: restaurant meal slots + B&B repair
Jun 15, 2026
d2fd692
Update listen submodule: mixed-mode transport + apostrophe fix + meal…
Jun 16, 2026
06cd041
Update listen submodule: OR_compound fix + meal-slot injection for MI…
Jun 16, 2026
0bb4141
Update listen submodule: required_cuisines + cuisine pinning + larger…
Jun 16, 2026
67b6b9c
Update listen submodule: ATTRACTION_COST budget-filter ordering + bud…
Jun 16, 2026
51d3681
Update listen submodule: cost-fail detection + assembler lunch-window…
Jun 16, 2026
3312563
Update listen submodule: MISSING_REQUIRED_RESTAURANT fixes + min_beds…
Jun 17, 2026
1a98734
Update listen submodule: required_dishes constraint + analyze_run cac…
Jun 17, 2026
f0ac7e9
Update listen submodule: content-based rank cache + lowercase type no…
Jun 17, 2026
1bf4f66
Update listen submodule: accommodation_proximity + min_visit_duration…
Jun 17, 2026
c944efb
Update listen submodule to 3410603: hotel pin fix + assembler fixes
Jun 18, 2026
42ba196
Update listen submodule: taxi fallback for early-morning departure ti…
Jun 18, 2026
949c999
Update listen submodule: >2km walk skip + hotel checkin cap + attr po…
Jun 18, 2026
1b1e5ed
Update listen submodule: dedup plan attractions + compound pin dedup;…
Jun 18, 2026
a3ac8bb
Add dynamic NL2SL few-shot corpus builder; update listen submodule
Jun 18, 2026
7d8d436
Update listen submodule: add 2nd morning slot + relax afternoon gate …
Jun 19, 2026
69a6005
Update listen submodule: add big_milp + cpsat solvers
Jun 19, 2026
4357d18
Update listen submodule: two-pass NL2SL with focused OR-group pass
Jun 20, 2026
b0eb477
Update listen submodule: fix MTZ subtour bug + attraction pool passth…
Jun 20, 2026
3b80b41
Update listen submodule: eval scoring CSV + CP-SAT improvements
Jun 21, 2026
61007fa
Update listen submodule: per-query eval_scores in result JSON
Jun 21, 2026
8df2c98
Add eval scripts, experiment logs, failing queries, and update listen…
Jun 21, 2026
efd7660
Update listen submodule: step-function DDR/DAV objective + day-1 brea…
Jun 21, 2026
e1fd4fd
Update listen submodule: fix travel-day/full-day attraction pool sepa…
Jun 21, 2026
8c085d2
Update listen submodule: fix transport position chain + extend ATT ce…
Jun 21, 2026
e34dc2b
Update listen submodule: Claude client + restaurant_budget + assemble…
Jun 21, 2026
19fed5e
Update listen submodule: recalibrate ATT weight + drop spurious objec…
Jun 22, 2026
5a88c91
Update listen submodule: 5-slot pre-breakfast + day-1 pool fix
Jun 22, 2026
ba6c3ad
Update listen submodule: fix end_ge+start_deadline routing infeasibility
Jun 22, 2026
7499c88
Update listen submodule: metro probe for inner_city_budget fix
Jun 22, 2026
ddc71ce
Update listen submodule: all-core CP-SAT + vectorized IC costs + taxi…
Jun 22, 2026
1cf9f7b
Update listen submodule: split departure-day attr pool to fix DAV
Jun 23, 2026
e0d002b
Update listen submodule: stacked DAV indicators + breakfast window fixes
Jun 23, 2026
b625cd6
Update listen submodule: nl2sl hotel/restaurant/transport regex rescu…
Jun 23, 2026
571300e
Update listen submodule: morning loop lunch-window guard
Jun 23, 2026
69480aa
Update listen submodule: travel-day IP extension for forced items + a…
Jun 23, 2026
9302e5b
Update listen submodule: assembler simplify + ticket-cost budget fixes
Jun 23, 2026
28d0fc2
Update listen submodule: wall-clock query timeout + cpsat_time_limit fix
Jun 24, 2026
1e087be
Update listen submodule: travel-day slots + overnight fixes + nl2sl v…
Jun 25, 2026
8ed1ce8
Update listen submodule: assembler overhaul + colgen; nesy_agent numb…
Jun 28, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
10 changes: 10 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,6 +1,11 @@
__pycache__/
.DS_Store
**/.DS_Store
.env
*.zip
__MACOSX/
**/__MACOSX/
*.egg-info/
.venv/
run_exp.sh
eval_exp.sh
Expand All @@ -16,3 +21,8 @@ uv.lock
.venv
agent_env/runs
agent_env/config.toml
rank_cache/
.claude/
LISTEN_v1/
LISTEN_v2/
LISTEN_v2 2/
4 changes: 4 additions & 0 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
[submodule "listen"]
path = listen
url = git@github.com:AdamJovine/LISTEN-active.git
branch = china
285 changes: 285 additions & 0 deletions AGENT_README.md

Large diffs are not rendered by default.

151 changes: 151 additions & 0 deletions BNB_EXPERIMENT_LOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,151 @@
# BnB Experiment Log — Session 2026-06-17

## Objective

Maximize pass rate on 95 originally-failing queries while keeping 39-passing sample at 100%.
All runs used `--rank-cache rank_cache` (no LLM calls, pure cache replay).

---

## Run Index

### Failing-set runs (95 queries originally failing at baseline)

| Commit | Run dir | Pass | Total | Rate | Notes |
|--------|---------|------|-------|------|-------|
| `6dfbe0a` | `results/failures_at_6dfbe0a/run_20260617_192902_bnb_groq` | 70 | 86 | 81% | Baseline (only 86 ran; 9 had errors/timeouts) |
| `04e2793` | `results/failures_at_04e2793/run_20260617_211323_bnb_groq` | 77 | 94 | 82% | After unuse_restaurant fix |
| `094f087` | `results/failures_at_094f087/run_20260617_213504_bnb_groq` | 80 | 95 | 84% | After poi_ordering fix |

### Passing-sample runs (39 queries that pass at baseline — regression guard)

| Commit | Run dir | Pass | Total | Rate |
|--------|---------|------|-------|------|
| `6dfbe0a` | `results/passing_sample_at_6dfbe0a/run_20260617_193914_bnb_groq` | 39 | 39 | 100% |
| `04e2793` | `results/passing_sample_assembler_fix/run_20260617_205749_bnb_groq` | 39 | 39 | 100% |
| `094f087` | `results/passing_sample_poiorder_fix/run_20260617_213300_bnb_groq` | 39 | 39 | 100% |

### Full-1000 runs

| Run dir | Pass | Total | Rate | Notes |
|---------|------|-------|------|-------|
| `results/FULL1000_BNB_MAX/run_20260616_184445_bnb_groq` | 905 | 1000 | 90.5% | Full dataset before this session's fixes |
| `results/run_20260617_180524_bnb_groq` | 132 | 152 | 86.8% | Partial run during session |

---

## Changes and Impact

### `04e2793` — assembler: unuse_restaurant + opentime check

**+1% on failing set** (70→77/94, 81%→82%)

**Root cause:** `next_restaurant()` marks a restaurant as "used" before `_meal_activity()` checks
if it can actually fit. If a restaurant (e.g. Chef's Wife, opens 11:00) was selected for a
lunch slot where arrival was after 14:00 (the lunch window close), it was permanently consumed
and unavailable for dinner.

**Fix 1:** Skip restaurants where `opentime >= meal_window_close` in `next_restaurant()`.
Added `_MEAL_WIN_CLOSE = {"breakfast": "09:00", "lunch": "14:00", "dinner": "20:00"}` dict
and pre-check in all three loop paths (priority, pool order, fallback).

**Fix 2:** Added `unuse_restaurant()` closure after `next_restaurant()` that returns a
restaurant to the pool (by object identity) when `_meal_activity()` or `_can_afford()` rejects
it. Added `else: unuse_restaurant(rest)` at all 6 meal-slot call sites in assembler.py.

**File:** `chinatravel_tpc/assembler.py`

---

### `094f087` — poi_ordering: ordered pin list + attr_pins re-sort

**+2% on failing set** (77→80/95, 82%→84%)

**Root cause 1:** `_parse_pinned_names()` used Python `set`, losing insertion order. For queries
requiring "Manchester United Dream Theater then Houhai", the set could return Houhai first,
prepending it before Manchester United in `attr_pins` and causing the poi_ordering constraint
to fail.

**Fix 1:** Changed `_parse_pinned_names()` to use ordered `list` (preserving insertion order
from nl2sl literals). `pins.setdefault(cat, [])` with dedup check.

**Fix 2:** Added explicit poi_ordering-aware `attr_pins` re-sort in `bnb_agent.py`. Parses
`idx_activity` patterns from `hard_logic_py` snippets to extract (before_name, after_name)
pairs, then swaps `attr_pins` entries that are in the wrong order.

**Files:** `chinatravel_tpc/agent.py`, `chinatravel_tpc/bnb_agent.py`

---

### `ab8f858` — agent: normalize interior whitespace in pin names

**Impact not yet measured** (committed 2026-06-17, run pending)

**Root cause:** Some DB hotel names have double spaces (e.g., "Atour X Hotel Shanghai Hongqiao
Airport Konggang Road") but nl2sl predicts single space. `_normalize_pin_name` only handled
parenthesis spacing.

**Fix:** Added `re.sub(r'\s+', ' ', name)` before the parenthesis normalization to collapse
all interior whitespace.

**Expected impact:** Resolves 1 of the 3 `required_hotel_name` failures (UID 20250322205724212077,
Atour X Hotel case). The other two hotel cases appear to be a different bug where the pinned
hotel is found by `_rows_for_pins` but never selected in the BnB skeleton phase.

**File:** `chinatravel_tpc/agent.py`

---

## Remaining 15 Failures at `094f087`

Run: `results/failures_at_094f087/run_20260617_213504_bnb_groq`

| UID | Failure type | Notes |
|-----|-------------|-------|
| 20250321040138918100 | `other` | Unknown constraint |
| 20250321114239878527 | `required_attraction_type` | |
| 20250322130400129262 | `required_hotel_feature` | |
| 20250322142408120417 | `OR_compound` | Hangzhou 4-day, sightseeing ≤200 OR single bed |
| 20250322161842269069 | `required_cuisine_type` | Japanese cuisine at airport hotel area |
| 20250322164349699070 | `required_restaurant` | |
| 20250322165301153800 | `required_cuisine_type` | Japanese cuisine at airport hotel area |
| 20250322171831366188 | `other` | |
| 20250322192555845893 | `poi_timing` | Peppa Pig 12:30-14:00 window |
| 20250322194001155137 | `required_hotel_name` | "Shanghai Lujiazui Babaiban Lan'ou Hotel" — pin found but skeleton ignores it |
| 20250322205724212077 | `required_hotel_name` | "Atour X Hotel Shanghai Hongqiao" — double-space mismatch (fixed in ab8f858) |
| 20250323010327713880 | `poi_timing` | Bistro Sola 17:00-18:00 window |
| 20250323031255302334 | `total_budget`, `poi_timing` | Blue Airflow Skydiving 14:20-15:50 window |
| 20250324082922869744 | `required_attraction` | |
| 20250324223916776179 | `required_hotel_name` | Third hotel pin case |

### Open investigations

**Hotel pin not selected in skeleton (2 cases: 20250322194001155137, 20250324223916776179):**
- `_rows_for_pins(hotel_df_full, ...)` finds the hotel (verified)
- `_prepend_pins(hotel_pins, hotel_winners)` puts it first in hotel_winners
- But skeleton output shows only other hotels (Waiting Hotel, Jinglai Hotel)
- Likely cause: `_find_feasible_skeletons` evaluates hotel against `hard_logic` and the
hotel fails some constraint (proximity? budget?), or `hotel_winners` is being re-sorted
after `_prepend_pins` runs
- Next step: add debug print inside `_find_feasible_skeletons` to see why pinned hotel is
rejected in the skeleton phase

**poi_timing (3 cases):**
- Assembler visits attraction outside its open window
- Need to check if `min_visit_duration` is consuming too much time before the timed attraction

**required_cuisine_type (2 cases):**
- Japanese restaurants near airport hotel areas not being selected
- May be a rank cache issue (ranked low) or proximity filter excluding them

---

## What Remains To Do

1. **Run failing set at `ab8f858`** to confirm double-space fix resolves UID 20250322205724212077
2. **Investigate hotel pin skeleton bypass** — why do pinned hotels get prepended to
`hotel_winners` but never appear in feasible skeletons (UIDs 20250322194001155137,
20250324223916776179)
3. **Investigate poi_timing failures** (3 cases) — assembler timing logic
4. **Investigate required_cuisine_type failures** (2 cases) — Japanese cuisine availability
5. **Run full 1000 at latest commit** after fixes accumulate to get updated full-dataset score
Empty file added FULL1000_BNB_MAX.log
Empty file.
Loading