diff --git a/docs/guides/fixture-corpus.md b/docs/guides/fixture-corpus.md index c0ab682d..a7462ef1 100644 --- a/docs/guides/fixture-corpus.md +++ b/docs/guides/fixture-corpus.md @@ -101,13 +101,15 @@ is unseen elsewhere. It is the conservative middle ground between the pair-set l ## Tooling -Two scripts implement the analysis. Both run through `uv`. +Three scripts, all run through `uv`. They are **report-only** — none mutates the +corpus or auto-recommends deletions (the plan-032 drop decision is already applied; +`scripts/build_fixture_corpus.py` is the one-time builder that produced the file). ### `scripts/profile_fixture_corpus.py` Parses every record and reports per-record provenance, `main_layout`, fired feature flags, the pair-set signature, parse errors, and corpus-wide rarity (which pairs / -layouts have only 1–2 carriers) plus subset-coverage. +layouts have only 1–2 carriers). ```bash uv run python scripts/profile_fixture_corpus.py # human-readable report @@ -116,15 +118,29 @@ uv run python scripts/profile_fixture_corpus.py --json # machine-readable ### `scripts/compare_drop_signatures.py` -Applies the signature readings above to the drop candidates, answers "does any -candidate have a unique signature?" under all three readings, and prints the **final -drop list under the distinct-type bar** — each dropped record annotated with the -surviving record that preserves its signature. +Reports the three signature readings and surfaces **distinct-type signature +clusters** (records sharing a component sequence) for human review. + +It does NOT recommend drops, by design. A shared `(type, sub_type)` signature is +blind to details-level structure: e.g. two `ai_overview/sectioned` records can +differ in section *count* (1 vs 3), and `test_ai_overview_legacy_sge.py` depends on +the multi-section one specifically. Always confirm at the details level — and check +the query-keyed tests — before treating a cluster as redundant. ```bash uv run python scripts/compare_drop_signatures.py ``` +### `scripts/verify_drops.py` + +Corpus-integrity guard: confirms the 8 plan-032 drops are absent, serp_ids are +unique, every record carries a `note`, the three witnessed layouts survive, and +every parsed `(type, sub_type)` has a carrier. Exits non-zero on failure (CI-usable). + +```bash +uv run python scripts/verify_drops.py +``` + ## The `note` field Surviving records should carry a `note` mirroring the curated format — a provenance @@ -135,19 +151,23 @@ clause plus what the record contributes: ``` e.g. `"Bulk corpus capture, v0.6.7a0 crawl 2026-02-06. Sole carrier of -knowledge/unit_converter."` Keep the provenance clause free of private-repo names -and any embedded crawl tokens / IPs (scrub `GOOGLE_ABUSE_EXEMPTION` URLs). +knowledge/unit_converter."` Keep the provenance clause free of private-repo names. +One record (`7049404a2dd6`) intentionally retains a `GOOGLE_ABUSE_EXEMPTION` URL +token as an artifact of how the crawler obtained an abuse exemption. ## Reproducing the corpus assessment ```bash -# 1. Profile every record and find unique contributors + redundancy +# 1. Profile every record: provenance, layouts, unique contributors, rarity uv run python scripts/profile_fixture_corpus.py -# 2. Compute the drop list under the distinct-type-order bar +# 2. Review distinct-type signature clusters (potential redundancy) uv run python scripts/compare_drop_signatures.py -# 3. After any change, confirm tests + that no unique pair/layout was lost +# 3. Corpus-integrity guard (drops absent, notes present, layouts + coverage intact) +uv run python scripts/verify_drops.py + +# 4. After any change, confirm the tests pass uv run pytest tests/test_parse_serp.py tests/test_parser_coverage.py \ tests/test_ai_overview_legacy_sge.py -q ``` diff --git a/docs/plans/032-fixture-corpus-notes-and-pruning.md b/docs/plans/032-fixture-corpus-notes-and-pruning.md index 1a8e562d..c1883772 100644 --- a/docs/plans/032-fixture-corpus-notes-and-pruning.md +++ b/docs/plans/032-fixture-corpus-notes-and-pruning.md @@ -1,9 +1,9 @@ --- -status: draft +status: done branch: feature/fixture-corpus-notes created: 2026-05-31T11:32:26-07:00 -completed: -pr: +completed: 2026-05-31T12:42:13-07:00 +pr: https://github.com/gitronald/WebSearcher/pull/143 --- # Annotate and prune the bulk SERP fixture corpus @@ -244,8 +244,9 @@ These records are committed to a **public** repo. While rewriting the files: - Drop the private-repo name from any note text (the curated v0.7.2 notes currently say "Captured via SearchAudits ...") — use a neutral provenance clause. -- Scrub the `GOOGLE_ABUSE_EXEMPTION` token + embedded IP from the `url` of - `serps-v0.6.7` record `7049404a2dd6` (mooted if that file is retired). +- **Decided: keep** the `GOOGLE_ABUSE_EXEMPTION` token (with its embedded client IP) + on record `7049404a2dd6` as a deliberate artifact — it documents how the crawler + obtained an abuse exemption, which is worth preserving. Not scrubbed. ### 5. Consolidate into `serps.json.bz2` and update loaders @@ -309,3 +310,46 @@ than retiring the file wholesale; consolidate into one version-less file. - **Should the freed budget be backfilled now** with new diverse captures, or left for a follow-up plan? Recommended: follow-up — keep this plan scoped to annotate + prune. + +## Log + +- **2026-05-31** — Implemented as specified across five commits: + - `ca43a31` — committed the methodology artifacts (fixture-corpus guide, this plan, + `scripts/profile_fixture_corpus.py`, `scripts/compare_drop_signatures.py`). + - `f5b6eb4` — recorded the Pass-B drop verification and added `scripts/verify_drops.py` + (per-drop coverage check against the whole surviving corpus: 0 drop-only items). + - `30dcaef` — consolidated all seven version-named fixtures into a single + `tests/fixtures/serps.json.bz2` (renamed from `serps-v0.6.8`, the six others deleted), + added a per-record `note` to every survivor via `scripts/build_fixture_corpus.py`, + dropped the 8 redundant records from §3, deleted their orphaned snapshots, and + `--snapshot-update`'d the ~22 newly-included parser-coverage/sge records. Repointed + the three test loaders at the single file. + - `7426854` — repointed `profile_fixture_corpus.py` / `compare_drop_signatures.py` at + `serps.json.bz2` and made them report-only. + - `4cc6aca` — clarified the specialized-components label in the cluster report. +- **Final corpus:** 80 records in one `serps.json.bz2` (~21 MB). The 8 drops match the + §3 table exactly (5 sky-blue, 1 streaming, 1 paragraph-query, 1 college). +- **Verify suite (§6) green** on close: `149 passed, 80 snapshots passed` across + `test_parse_serp`, `test_parser_coverage`, `test_ai_overview_legacy_sge`, + `test_extractor_serp_features`; `test_features_expose_main_layout` included, no + orphaned or missing snapshots, every single-carrier `(type, sub_type)` pair retained. +- **2026-05-31** — Closed: PR #143 merged into `feature/v0.9.0` (the 0.9.0 integration + branch). + +## Retrospective + +- The two-pass drop policy (mechanical signature screen, then adversarial value-level + `details` rescue) earned its keep: the unordered pair-set screen flagged ~25 subset + candidates, but the decided distinct-type-order bar plus the rescue pass cut that to + exactly 8 — protecting 17 records the cruder screen would have over-dropped. +- The one genuinely load-bearing constraint was mechanical, not structural: keeping + `f6fae1c9a96e` because it is the sole `standard-overview` carrier *within the + `serps-v*` glob* that `test_features_expose_main_layout` asserts against. Worth + re-checking that assertion's source set first on any future fixture surgery. +- Consolidating to one version-less file widened `test_parse_serp` from the `serps-v*` + subset to the whole corpus (+~22 snapshots). That was the right default — provenance + already lives per-record in the `version` field, so the filename carried no + information the JSON didn't. +- This branch sat fully implemented but with `status: draft` and no PR — the work and + the bookkeeping had drifted apart. Closing it promptly after the last impl commit + would have avoided the "did we execute this?" ambiguity. diff --git a/scripts/build_fixture_corpus.py b/scripts/build_fixture_corpus.py new file mode 100644 index 00000000..5844cf73 --- /dev/null +++ b/scripts/build_fixture_corpus.py @@ -0,0 +1,154 @@ +"""Build the consolidated SERP fixture corpus: tests/fixtures/serps.json.bz2. + +Merges the seven version-named bz2 fixtures into one, dropping the 8 verified- +redundant records (plan 032), scrubbing private-repo names from notes, and adding a +`note` to every survivor. Notes are generated from the POST-DROP corpus so "sole +carrier" claims are accurate; curated notes are preserved (only scrubbed). The one +surviving google_abuse exemption token is kept intact as a deliberate artifact. The +HTML field is written last on each line for readability. + +Usage: + uv run python scripts/build_fixture_corpus.py # write the file + uv run python scripts/build_fixture_corpus.py --check # report, do not write +""" + +import argparse +import bz2 +from collections import Counter, defaultdict +from pathlib import Path + +import orjson + +import WebSearcher as ws + +FIXTURES_DIR = Path("tests/fixtures") +OUT = FIXTURES_DIR / "serps.json.bz2" + +# Input files in the order they should appear in the consolidated file. +SOURCES = [ + "serps-parser-coverage.json.bz2", + "serps-sge-2024.json.bz2", + "serps-v0.6.7.json.bz2", + "serps-v0.6.8.json.bz2", + "serps-v0.7.2-ads.json.bz2", + "serps-v0.7.2-jobs.json.bz2", + "serps-v0.7.2-knowledge-subcards.json.bz2", +] + +DROPS = { + "97404b7b7c61", + "45b6e019bfa2", + "c9ab650f5bda", + "032572e185d3", + "be99c971b8f7", + "cad43c3268a8", + "3c09a0f0c92f", + "984065877aad", +} + +UBIQUITOUS = ("general/", "searches_related/", "people_also_ask/") + + +def scrub_note(note: str) -> str: + return note.replace("SearchAudits directives crawl", "a directives crawl") + + +def signature(parsed: dict) -> list[str]: + pairs = sorted( + {(r["type"], r["sub_type"]) for r in parsed["results"]}, + key=lambda ts: (ts[0] or "", ts[1] or ""), + ) + return [f"{t}/{s}" for t, s in pairs] + + +def load_sources() -> list[dict]: + survivors = [] + for name in SOURCES: + path = FIXTURES_DIR / name + with bz2.open(path, "rt") as f: + for line in f: + r = orjson.loads(line) + if r["serp_id"][:12] in DROPS: + continue + r["_parsed"] = ws.parse_serp(r["html"]) + r["_sig"] = signature(r["_parsed"]) + r["_layout"] = r["_parsed"]["features"].get("main_layout") + survivors.append(r) + return survivors + + +def build_notes(survivors: list[dict]) -> None: + pair_carriers: dict[str, set] = defaultdict(set) + layout_carriers: dict[str, set] = defaultdict(set) + for r in survivors: + sid = r["serp_id"][:12] + for p in r["_sig"]: + pair_carriers[p].add(sid) + layout_carriers[str(r["_layout"])].add(sid) + + for r in survivors: + if r.get("note"): # curated -> preserve, scrub only + r["note"] = scrub_note(r["note"]) + continue + sid = r["serp_id"][:12] + unique = [p for p in r["_sig"] if len(pair_carriers[p]) == 1] + rare = [p for p in r["_sig"] if len(pair_carriers[p]) == 2] + layout = r["_layout"] + clauses = [] + if unique: + clauses.append("sole carrier of " + ", ".join(unique)) + if layout and layout != "standard" and len(layout_carriers[str(layout)]) == 1: + clauses.append(f"only {layout} layout in the corpus") + if not unique and rare: + clauses.append("one of two carriers of " + ", ".join(rare)) + if not clauses: + notable = [p for p in r["_sig"] if not p.startswith(UBIQUITOUS)] + if notable: + clauses.append("coverage for " + ", ".join(notable[:4])) + else: + clauses.append("standard organic-results SERP") + contribution = "; ".join(clauses) + contribution = contribution[0].upper() + contribution[1:] + "." + prov = f"Corpus capture, WebSearcher {r.get('version')}, {r.get('timestamp', '')[:10]}." + r["note"] = f"{prov} {contribution}" + + +def emit(survivors: list[dict]) -> None: + with bz2.open(OUT, "wt") as f: + for r in survivors: + rec = {k: v for k, v in r.items() if not k.startswith("_")} + html = rec.pop("html") + note = rec.pop("note") + rec["note"] = note + rec["html"] = html # html last for readability + f.write(orjson.dumps(rec).decode() + "\n") + + +def main() -> None: + ap = argparse.ArgumentParser() + ap.add_argument("--check", action="store_true", help="report only, do not write") + args = ap.parse_args() + + survivors = load_sources() + build_notes(survivors) + + print(f"survivors: {len(survivors)} (dropped {len(DROPS)})") + print(f"all have notes: {all(r.get('note') for r in survivors)}") + print(f"versions: {dict(Counter(r.get('version') for r in survivors))}") + print(f"layouts: {dict(Counter(str(r['_layout']) for r in survivors))}") + tokens = [r["serp_id"][:12] for r in survivors if "google_abuse" in r.get("url", "")] + print(f"urls with google_abuse token (kept as artifact): {tokens}") + print("\nsample generated notes:") + for r in survivors[:3] + survivors[-4:]: + print(f" [{r['serp_id'][:12]}] {r['note']}") + + if args.check: + print("\n--check: not writing.") + return + emit(survivors) + size = OUT.stat().st_size + print(f"\nwrote {OUT} ({size / 1e6:.1f} MB)") + + +if __name__ == "__main__": + main() diff --git a/scripts/compare_drop_signatures.py b/scripts/compare_drop_signatures.py index a867cb9d..b244eda9 100644 --- a/scripts/compare_drop_signatures.py +++ b/scripts/compare_drop_signatures.py @@ -1,21 +1,22 @@ -"""Compare drop-candidate SERPs under an ORDERED component-name signature. +"""Report SERP signature redundancy across the consolidated corpus. -The corpus profiler (`profile_fixture_corpus.py`) screens redundancy with an -*unordered set* of (type, sub_type) pairs and a subset test. This script applies a -stricter lens -- the user's definition of a "SERP signature": the components listed -**in SERP order**, concatenated into a string. Order and multiplicity both count, so -[general, general, knowledge] != [knowledge, general]. +Reads tests/fixtures/serps.json.bz2 and, for every record, builds the components in +SERP order (one entry per cmpt_rank, from its sub_rank==0 row) under three readings: -For every record we build, ordered by cmpt_rank (one entry per component, taken from -its sub_rank==0 row): + sig_type = "knowledge > general > general > people_also_ask > ..." (incl. repeats) + sig_full = "knowledge[panel] > general > ..." (incl. sub_type) + sig_distinct = "knowledge > general > people_also_ask > ..." (repeats collapsed) - sig_type = "knowledge > general > general > people_also_ask > ..." - sig_full = "knowledge[panel] > general > general > people_also_ask > ..." +It then surfaces *clusters* of records that share a distinct-type signature -- i.e. +records whose component sequence is structurally similar -- as candidates a human +might review for redundancy. -Then, for each drop candidate (snapshot-only files, set-subset of a keeper, no -unique pair, excluding the designated keepers), we report whether its ordered -signature is duplicated **anywhere else in the corpus** -- and show it side by side -with the keeper it was matched to, so redundancy can be eyeballed. +IMPORTANT: this is a REPORT, not a drop recommender. A shared (type, sub_type) +signature does NOT prove two records are interchangeable: the signature is blind to +details-level structure. For example, two `ai_overview/sectioned` records can differ +in section COUNT (1 vs 3 sections), and a query-keyed test may depend on the +multi-section one specifically (see test_ai_overview_legacy_sge.py). Always confirm +at the details level (and check the query-keyed tests) before removing anything. Usage: uv run python scripts/compare_drop_signatures.py @@ -29,28 +30,15 @@ import WebSearcher as ws -FIXTURES_DIR = Path("tests/fixtures") - -# Records kept by explicit decision despite a subset signature: -# sky-blue representatives (keep 2) + the sole standard-overview in the snapshot set -KEEP_OVERRIDE = {"aa594f199c3d", "6aa70651b0cd", "f6fae1c9a96e"} - -SNAPSHOT_FILES = {"serps-v0.6.7.json.bz2", "serps-v0.6.8.json.bz2"} +FIXTURE = Path("tests/fixtures") / "serps.json.bz2" def load_all() -> list[dict]: - out = [] - for p in sorted(FIXTURES_DIR.glob("*.json.bz2")): - with bz2.open(p, "rt") as f: - for line in f: - r = orjson.loads(line) - r["_file"] = p.name - out.append(r) - return out + with bz2.open(FIXTURE, "rt") as f: + return [orjson.loads(line) for line in f] def ordered_components(parsed: dict) -> list[tuple[str, str | None]]: - """One (type, sub_type) per component, in cmpt_rank order (sub_rank==0 row).""" by_cmpt: dict[int, dict] = {} for r in parsed["results"]: c = r["cmpt_rank"] @@ -68,7 +56,6 @@ def sig_full(comps) -> str: def sig_distinct(comps) -> str: - """Distinct component TYPES in first-appearance order (collapse repeats).""" seen, out = set(), [] for t, _ in comps: if t not in seen: @@ -77,164 +64,55 @@ def sig_distinct(comps) -> str: return " > ".join(out) -def diff_marks(a: str, b: str) -> str: - """Human note comparing two ' > '-joined signatures.""" - A, B = a.split(" > "), b.split(" > ") - if A == B: - return "IDENTICAL ordered signature" - ca, cb = Counter(A), Counter(B) - only_a = list((ca - cb).elements()) - only_b = list((cb - ca).elements()) - parts = [] - if only_a: - parts.append(f"candidate has extra: {only_a}") - if only_b: - parts.append(f"keeper has extra: {only_b}") - if not parts: - parts.append("same multiset, different ORDER") - return "; ".join(parts) - - def main() -> None: - records = load_all() - - # Per-record profile prof = {} - for r in records: - parsed = ws.parse_serp(r["html"]) - comps = ordered_components(parsed) + for r in load_all(): + comps = ordered_components(ws.parse_serp(r["html"])) prof[r["serp_id"][:12]] = { "short": r["serp_id"][:12], - "file": r["_file"], "qry": r.get("qry", ""), - "comps": comps, - "pair_set": {(t, s) for t, s in comps}, "rich_types": {t for t, _ in comps if t != "general"}, "sig_type": sig_type(comps), "sig_full": sig_full(comps), "sig_distinct": sig_distinct(comps), - "n": len(comps), } - # Corpus-wide ordered-signature frequency (across ALL files) type_freq = Counter(p["sig_type"] for p in prof.values()) full_freq = Counter(p["sig_full"] for p in prof.values()) distinct_freq = Counter(p["sig_distinct"] for p in prof.values()) - type_carriers = defaultdict(list) - full_carriers = defaultdict(list) - for p in prof.values(): - type_carriers[p["sig_type"]].append(p["short"]) - full_carriers[p["sig_full"]].append(p["short"]) - # Unique (type/sub_type) pairs -> keeper screen (matches the profiler) - pair_carriers = defaultdict(set) - for p in prof.values(): - for pair in p["pair_set"]: - pair_carriers[pair].add(p["short"]) - - # Drop candidates: snapshot-only, subset of another's pair_set, no unique pair, - # not a designated keeper. - sig_sets = {s: p["pair_set"] for s, p in prof.items()} - drops = [] - for s, p in prof.items(): - if p["file"] not in SNAPSHOT_FILES or s in KEEP_OVERRIDE: - continue - if any(len(pair_carriers[pr]) == 1 for pr in p["pair_set"]): - continue # carries a unique pair -> keeper - supersets = [q for q in prof if q != s and sig_sets[s] < sig_sets[q]] - equals = [q for q in prof if q != s and sig_sets[s] == sig_sets[q]] - if supersets or equals: - p["match"] = supersets or equals - drops.append(p) - - # Report -------------------------------------------------------------- - print(f"Corpus: {len(prof)} records. Drop candidates (set-subset screen): {len(drops)}\n") - - uniq_type = [p for p in drops if type_freq[p["sig_type"]] == 1] - uniq_full = [p for p in drops if full_freq[p["sig_full"]] == 1] - uniq_dist = [p for p in drops if distinct_freq[p["sig_distinct"]] == 1] - print("ANSWER TO 'do any drops have a UNIQUE serp signature?' (3 readings)\n") + print(f"Corpus: {len(prof)} records.\n") + print("Signature uniqueness across the corpus (3 readings):") print( - f" A. ordered, every component incl. repeats (TYPE): {len(uniq_type):2d}/{len(drops)} unique" + f" ordered incl. repeats (type): {sum(1 for p in prof.values() if type_freq[p['sig_type']] == 1)}/{len(prof)} unique" ) print( - f" B. ordered, incl. repeats + sub_type (TYPE[sub_type]): {len(uniq_full):2d}/{len(drops)} unique" + f" ordered incl. repeats + sub_type: {sum(1 for p in prof.values() if full_freq[p['sig_full']] == 1)}/{len(prof)} unique" ) print( - f" C. distinct types, first-appearance order (no repeats): {len(uniq_dist):2d}/{len(drops)} unique" + f" distinct types, order, no repeats: {sum(1 for p in prof.values() if distinct_freq[p['sig_distinct']] == 1)}/{len(prof)} unique\n" ) - print(" (a unique STRING is not unique parser COVERAGE -- see rich-component column)\n") - print("=" * 100) - print("COMPACT SUMMARY (rich = non-'general' component types absent from matched keeper)") - print("=" * 100) - print(f"{'serp_id':13s} {'qry':34s} {'A':3s} {'B':3s} {'C':3s} rich-components-keeper-lacks") - for p in sorted(drops, key=lambda x: (x["file"], x["qry"])): - keeper = prof[p["match"][0]] - rich_extra = sorted(p["rich_types"] - keeper["rich_types"]) - a = "U" if type_freq[p["sig_type"]] == 1 else f"x{type_freq[p['sig_type']]}" - b = "U" if full_freq[p["sig_full"]] == 1 else f"x{full_freq[p['sig_full']]}" - c = "U" if distinct_freq[p["sig_distinct"]] == 1 else f"x{distinct_freq[p['sig_distinct']]}" - verdict = rich_extra if rich_extra else "(none -- only differs by general count/order)" - print(f"{p['short']:13s} {p['qry'][:34]:34s} {a:3s} {b:3s} {c:3s} {verdict}") - print() - - # -- Apply chosen bar: DISTINCT-TYPE ORDER ----------------------------- - # A candidate may be dropped only if its distinct-type ordered signature is - # still carried by a SURVIVING record. Records with a unique sig_distinct are - # protected; within a cluster sharing a sig_distinct that no non-candidate - # carries, keep one representative (the fullest by component count). - cand_ids = {p["short"] for p in drops} - noncand_sigs = {p["sig_distinct"] for s, p in prof.items() if s not in cand_ids} - final_drop, protected = [], [] - by_sig = defaultdict(list) - for p in drops: - by_sig[p["sig_distinct"]].append(p) - for sig, members in by_sig.items(): - members.sort(key=lambda x: -x["n"]) # fullest first - if distinct_freq[sig] == 1: - protected.extend((m, "unique distinct-type signature") for m in members) - elif sig in noncand_sigs: - survivors = sorted( - s for s, pp in prof.items() if pp["sig_distinct"] == sig and s not in cand_ids - ) - final_drop.extend((m, survivors) for m in members) - else: - protected.append((members[0], "representative of shared signature")) - final_drop.extend((m, [members[0]["short"]]) for m in members[1:]) + clusters = defaultdict(list) + for p in prof.values(): + clusters[p["sig_distinct"]].append(p) + shared = {sig: members for sig, members in clusters.items() if len(members) > 1} print("=" * 100) print( - f"FINAL DROP LIST under DISTINCT-TYPE bar: {len(final_drop)} drop / {len(protected)} protected (of {len(drops)} candidates)" + f"DISTINCT-TYPE SIGNATURE CLUSTERS (>=2 records share a component sequence): {len(shared)}" ) + print("REVIEW ONLY -- confirm details-level structure + query-keyed tests before removing any.") + print("('general' = plain organic results, on nearly every SERP, so it carries no") + print(" signal; the line below names only the specialized component types in the") + print(" shared sequence -- the distinctive parser paths worth comparing.)") print("=" * 100) - print("DROP (distinct-type signature survives elsewhere):") - for p, preserved in sorted(final_drop, key=lambda x: x[0]["qry"]): - print(f" {p['short']} {p['qry'][:42]:42s} preserved by: {preserved}") - print("\nPROTECTED (kept despite set-subset screen):") - for p, why in sorted(protected, key=lambda x: x[0]["qry"]): - print(f" {p['short']} {p['qry'][:42]:42s} {why}") - print() - - print("=" * 100) - print("PER DROP-CANDIDATE COMPARISON (ordered signature vs matched keeper)") - print("=" * 100) - for p in sorted(drops, key=lambda x: (x["file"], x["qry"])): - keeper = prof[p["match"][0]] - utype = ( - "UNIQUE" - if type_freq[p["sig_type"]] == 1 - else f"x{type_freq[p['sig_type']]} ({type_carriers[p['sig_type']]})" - ) - print( - f"\n{p['short']} {p['file'].replace('serps-', '').replace('.json.bz2', '')} {p['qry']!r}" - ) - print(f" ordered TYPE sig [{utype}]:") - print(f" {p['sig_type']}") - print(f" full sig: {p['sig_full']}") - print(f" matched keeper {keeper['short']} {keeper['qry']!r}:") - print(f" {keeper['sig_type']}") - print(f" delta: {diff_marks(p['sig_full'], keeper['sig_full'])}") + for sig, members in sorted(shared.items(), key=lambda kv: -len(kv[1])): + specialized = sorted({rt for m in members for rt in m["rich_types"]}) + print(f"\n[{len(members)} records] {sig}") + print(f" specialized (non-'general') component types: {specialized or '(none)'}") + for m in members: + print(f" {m['short']} {m['qry'][:60]!r}") if __name__ == "__main__": diff --git a/scripts/profile_fixture_corpus.py b/scripts/profile_fixture_corpus.py index a4b11944..36e35ce1 100644 --- a/scripts/profile_fixture_corpus.py +++ b/scripts/profile_fixture_corpus.py @@ -29,14 +29,18 @@ import WebSearcher as ws FIXTURES_DIR = Path("tests/fixtures") -# Bulk fixtures = everything except the curated single-component v0.7.2 set, -# which already carries a per-record note. -BULK_GLOBS = [ - "serps-v0.6.7.json.bz2", - "serps-v0.6.8.json.bz2", - "serps-parser-coverage.json.bz2", - "serps-sge-2024.json.bz2", -] +# The consolidated corpus (plan 032). Falls back to the pre-consolidation bulk +# files if the single file is absent. +BULK_GLOBS = ( + ["serps.json.bz2"] + if (FIXTURES_DIR / "serps.json.bz2").exists() + else [ + "serps-v0.6.7.json.bz2", + "serps-v0.6.8.json.bz2", + "serps-parser-coverage.json.bz2", + "serps-sge-2024.json.bz2", + ] +) def load(path: Path) -> list[dict]: diff --git a/scripts/verify_drops.py b/scripts/verify_drops.py index 393bcfae..eba746f7 100644 --- a/scripts/verify_drops.py +++ b/scripts/verify_drops.py @@ -1,87 +1,66 @@ -"""Pass B: confirm the 8 proposed fixture drops lose no parser coverage. +"""Verify the consolidated SERP corpus integrity (post plan-032). -For each drop, we ask the strict question: is every bit of parser coverage it -exercises ALSO present in at least one SURVIVING record (the whole corpus minus all 8 -drops)? Coverage is measured at four granularities: +Confirms the plan-032 pruning + consolidation landed cleanly in +tests/fixtures/serps.json.bz2: - * component = (type, sub_type) - * detail_shape= (type, sub_type, sorted(details.keys()), details.get("type")) - * error = (type, sub_type, error) - * fields = (type, sub_type, which of title/url/text/cite are non-null) + * the 8 dropped records are absent + * serp_ids are unique + * every record carries a non-empty `note` + * the three witnessed layouts are still present (standard / standard-overview / + standard-airfares) -- the set test_features_expose_main_layout pins + * every (type, sub_type) pair the corpus parses has at least one carrier (sanity) -`drop_only` = coverage tuples present in the drop but in NO survivor. A non-empty -drop_only is a rescue flag. +Exits non-zero if any check fails, so it can double as a CI guard. Usage: - uv run python scripts/verify_drops.py # coverage diff for all 8 - uv run python scripts/verify_drops.py --dump ID # readable results+details dump + uv run python scripts/verify_drops.py + uv run python scripts/verify_drops.py --dump ID # readable results+details dump """ import argparse import bz2 +import sys +from collections import Counter from pathlib import Path import orjson import WebSearcher as ws -FIXTURES_DIR = Path("tests/fixtures") - -# drop -> the record whose distinct-type signature preserves it (for reference only; -# the coverage test below is against the WHOLE surviving corpus, not just this one). -DROPS = { - "97404b7b7c61": "7049404a2dd6", - "45b6e019bfa2": "7049404a2dd6", - "c9ab650f5bda": "7049404a2dd6", - "032572e185d3": "7049404a2dd6", - "be99c971b8f7": "7049404a2dd6", - "cad43c3268a8": "9ed1baa7715d", - "3c09a0f0c92f": "f006c9318116", - "984065877aad": "9a7e39d95bf0", +FIXTURE = Path("tests/fixtures") / "serps.json.bz2" + +# The 8 records removed by plan 032 (must no longer be present). +PLAN032_DROPS = { + "97404b7b7c61", + "45b6e019bfa2", + "c9ab650f5bda", + "032572e185d3", + "be99c971b8f7", + "cad43c3268a8", + "3c09a0f0c92f", + "984065877aad", } +REQUIRED_LAYOUTS = {"standard", "standard-overview", "standard-airfares"} -def load_all() -> dict[str, dict]: - recs = {} - for p in sorted(FIXTURES_DIR.glob("*.json.bz2")): - with bz2.open(p, "rt") as f: - for line in f: - r = orjson.loads(line) - recs[r["serp_id"][:12]] = r - return recs - - -def coverage(parsed: dict) -> dict[str, set]: - comp, shape, err, fields = set(), set(), set(), set() - for r in parsed["results"]: - ts = (r["type"], r["sub_type"]) - comp.add(ts) - d = r.get("details") - if isinstance(d, dict): - shape.add((ts, tuple(sorted(d.keys())), d.get("type"))) - elif isinstance(d, list): - shape.add((ts, ("",), None)) - if r.get("error") is not None: - err.add((ts, r["error"])) - present = tuple(k for k in ("title", "url", "text", "cite") if r.get(k) is not None) - fields.add((ts, present)) - return {"component": comp, "detail_shape": shape, "error": err, "fields": fields} +def load() -> list[dict]: + with bz2.open(FIXTURE, "rt") as f: + return [orjson.loads(line) for line in f] def dump(rec: dict) -> None: parsed = ws.parse_serp(rec["html"]) print(f"# {rec['serp_id'][:12]} qry={rec.get('qry')!r} v{rec.get('version')}") print( - f"# {len(parsed['results'])} results / features.main_layout={parsed['features'].get('main_layout')}\n" + f"# {len(parsed['results'])} results / main_layout={parsed['features'].get('main_layout')}\n" ) for r in parsed["results"]: print( f"[{r['cmpt_rank']}.{r['sub_rank']}] {r['type']}/{r['sub_type']} " f"title={r['title']!r} url={(r['url'] or '')[:60]!r} err={r['error']!r}" ) - d = r.get("details") - if d: - print(f" details={orjson.dumps(d).decode()[:400]}") + if r.get("details"): + print(f" details={orjson.dumps(r['details']).decode()[:400]}") def main() -> None: @@ -89,38 +68,47 @@ def main() -> None: ap.add_argument("--dump", metavar="ID") args = ap.parse_args() - recs = load_all() + recs = load() + by_id = {r["serp_id"][:12]: r for r in recs} if args.dump: - dump(recs[args.dump]) + dump(by_id[args.dump]) return - # Surviving corpus = everything except the 8 drops. - survivor_ids = [s for s in recs if s not in DROPS] - survivor_cov = {"component": set(), "detail_shape": set(), "error": set(), "fields": set()} - for s in survivor_ids: - c = coverage(ws.parse_serp(recs[s]["html"])) - for k in survivor_cov: - survivor_cov[k] |= c[k] - - print(f"Survivors: {len(survivor_ids)} records. Checking {len(DROPS)} drops.\n") - all_clear = True - for drop, preserver in DROPS.items(): - c = coverage(ws.parse_serp(recs[drop]["html"])) - drop_only = {k: sorted(map(str, c[k] - survivor_cov[k])) for k in c} - n_only = sum(len(v) for v in drop_only.values()) - status = "SAFE" if n_only == 0 else f"RESCUE ({n_only} drop-only)" - if n_only: - all_clear = False - print(f"{drop} {recs[drop].get('qry')[:38]!r:40s} -> survives via {preserver}: {status}") - for k, v in drop_only.items(): - if v: - print(f" drop-only {k}:") - for item in v: - print(f" {item}") - print( - f"\n{'ALL 8 SAFE — no coverage lost vs surviving corpus.' if all_clear else 'RESCUES NEEDED — see above.'}" + checks: list[tuple[str, bool, str]] = [] + + present_drops = sorted(PLAN032_DROPS & set(by_id)) + checks.append(("8 plan-032 drops absent", not present_drops, f"still present: {present_drops}")) + + checks.append( + ( + "serp_ids unique", + len(by_id) == len(recs), + f"{len(recs)} records, {len(by_id)} unique ids", + ) + ) + + no_note = [r["serp_id"][:12] for r in recs if not r.get("note")] + checks.append(("every record has a note", not no_note, f"missing note: {no_note}")) + + layouts = Counter(ws.parse_serp(r["html"])["features"].get("main_layout") for r in recs) + missing_layouts = REQUIRED_LAYOUTS - set(layouts) + checks.append(("required layouts present", not missing_layouts, f"missing: {missing_layouts}")) + + pairs = Counter( + (res["type"], res["sub_type"]) for r in recs for res in ws.parse_serp(r["html"])["results"] ) + checks.append(("every (type,sub_type) has a carrier", all(c > 0 for c in pairs.values()), "")) + + print(f"Corpus: {len(recs)} records, {len(pairs)} distinct (type,sub_type) pairs") + print(f"layouts: {dict(layouts)}\n") + ok = True + for name, passed, detail in checks: + ok &= passed + mark = "PASS" if passed else "FAIL" + print(f" [{mark}] {name}" + (f" -- {detail}" if not passed else "")) + print(f"\n{'ALL CHECKS PASSED' if ok else 'INTEGRITY CHECK FAILED'}") + sys.exit(0 if ok else 1) if __name__ == "__main__": diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[032572e185d3].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[032572e185d3].json deleted file mode 100644 index c13b978a..00000000 --- a/tests/__snapshots__/test_parse_serp/test_parse_serp[032572e185d3].json +++ /dev/null @@ -1,530 +0,0 @@ -{ - "features": { - "captcha": false, - "infinity_scroll": false, - "language": "en", - "main_layout": "standard", - "notice_no_results": false, - "notice_server_error": false, - "notice_shortened_query": false, - "overlay_precise_location": false, - "result_estimate_count": 2980000000.0, - "result_estimate_time": 0.26 - }, - "results": [ - { - "cite": null, - "cmpt_rank": 0, - "details": { - "citations": [ - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "1", - "3", - "4", - "2", - "7" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "6", - "12", - "15", - "8", - "5", - "9" - ] - } - ], - "sources": [ - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.weather.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Weather Service (.gov)", - "snippet": "While all colors are scattered by air molecules, violet and blue are scattered most. The sky looks blue, not violet, because our e...", - "source_id": "2", - "title": "Why Is The Sky Blue? - National Weather Service", - "url": "https://www.weather.gov/fgz/SkyBlue" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.nesdis.noaa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Environmental Satellite, Data, and Information Service NESDIS (.gov)", - "snippet": "Gases and particles in Earth's atmosphere scatter sunlight in all directions. Blue light is scattered more than other colors becau...", - "source_id": "1", - "title": "Why Is the Sky Blue? - NESDIS - NOAA", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue#:~:text=Gases%20and%20particles%20in%20Earth's%20atmosphere%20scatter,a%20blue%20sky%20most%20of%20the%20time." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.facebook.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Facebook", - "snippet": "The sky appears blue because of a phenomenon called Rayleigh scattering. Sunlight is made up of all the colors of the rainbow, eac...", - "source_id": "4", - "title": "Why is the sky blue? It's an age old question that actually has a very simple answer The sun emits a white light that consists of all the colors of the rainbow At about 18 miles above the earth the light begins to interact with air molecules Those air molecules are the perfect size to scatter the blue light wavelengths, but all other colors continue down to the surface The blue light is scattered from molecule to molecule and reflected at you And now you know 🙃", - "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://morgridge.org&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Morgridge Institute for Research -", - "snippet": "The sky appears blue because of the scattering of blue light by the oxygen and nitrogen in our atmosphere. Light travels in waves,", - "source_id": "7", - "title": "Why is the sky blue? - Morgridge Institute for Research", - "url": "https://morgridge.org/blue-sky/why-is-the-sky-blue/" - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.reddit.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Reddit", - "snippet": "The sky is blue because blue light gets scattered the most during the day. Rayleigh scattering states that the amount of scatterin...", - "source_id": "8", - "title": "Why is the sky blue? Do I understand it correctly: : r/askscience", - "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://math.ucr.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "University of California, Riverside", - "snippet": "We have three types of colour receptors, or cones, in our retina. They are called red, blue and green because they respond most st...", - "source_id": "12", - "title": "Why is the sky blue? - UCR Math", - "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html#:~:text=We%20have%20three%20types%20of%20colour%20receptors%2C,visual%20system%20constructs%20the%20colours%20we%20see." - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://kids.nationalgeographic.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Geographic Kids", - "snippet": "The sky is blue because of the scattering of light in the atmosphere. The sun's light is made up of all the colors of the rainbow,", - "source_id": "6", - "title": "Why is the sky blue? | National Geographic Kids", - "url": "https://kids.nationalgeographic.com/books/article/sky" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.uu.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Union University", - "snippet": "The sky appears blue because of light scattering. Light waves interact with air molecules, and the air particles quickly re-radiat...", - "source_id": "5", - "title": "Why is the sky blue on Earth, but black in space or on the Moon? | Science Guys | Union University, a Christian College in Tennessee", - "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.britannica.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Britannica", - "snippet": "One of the perennial questions of childhood is “Why is the sky blue?” You may have asked this as a child, or you may have a child ...", - "source_id": "15", - "title": "Why Is the Sky Blue? | Britannica", - "url": "https://www.britannica.com/story/why-is-the-sky-blue" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.montrealsciencecentre.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Montreal Science Centre", - "snippet": "Wrapped around the Earth, the atmosphere is made up of a thin layer of gases, mainly nitrogen and oxygen. This is where the air we...", - "source_id": "9", - "title": "Why is the Sky Blue? - Montreal Science Centre", - "url": "https://www.montrealsciencecentre.com/blog/why-the-sky-blue" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://plus.nasa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "NASA+ (.gov)", - "snippet": "When sunlight enters Earth's atmosphere, it encounters gases that scatter the light in all directions. The shorter, choppier waves...", - "source_id": "3", - "title": "Space Place in a Snap: Why Is the Sky Blue?", - "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/#:~:text=When%20sunlight%20enters%20Earth's%20atmosphere%2C%20it%20encounters,sky%20appears%20blue%20on%20a%20sunny%20day." - } - ], - "type": "ai_overview" - }, - "error": null, - "section": "main", - "serp_rank": 0, - "sub_rank": 0, - "sub_type": "flat", - "text": "The sky appears blue due to a phenomenon called Rayleigh scattering , where molecules and small particles in Earth's atmosphere scatter shorter, blue-wavelength sunlight in all directions more efficiently than other colors. Because this scattered blue light is distributed across the sky, our eyes perceive it as blue during the day. Key Details About Sky Color: Sunlight Composition: Sunlight appears white but is actually made up of all the colors of the rainbow. Rayleigh Scattering: As sunlight hits gas molecules (mostly nitrogen and oxygen) in the atmosphere, the light scatters. Blue light travels in smaller, shorter waves, causing it to scatter roughly four times more than red light . Why Not Violet? While violet light has an even shorter wavelength, the sky looks blue because the sun emits more blue light than violet, and human eyes are more sensitive to blue light. Sunsets/Sunrises: When the sun is low on the horizon, light passes through more atmosphere. This causes the blue light to scatter away entirely, leaving the longer red, yellow, and orange wavelengths to reach our eyes. Space is Black: Because space lacks an atmosphere and particles to scatter sunlight, the sky appears black, as seen from the Moon. The sky appears paler near the horizon because the scattered light must pass through more air, allowing more of the blue light to be scattered away before reaching the eyes.", - "title": null, - "type": "ai_overview", - "url": null - }, - { - "cite": null, - "cmpt_rank": 1, - "details": { - "items": [ - "Why is the sky blue short answer?", - "What is the true color of our sky?", - "How to explain to a kid why the sky is blue?", - "Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar" - ], - "type": "text" - }, - "error": null, - "section": "main", - "serp_rank": 1, - "sub_rank": 0, - "sub_type": null, - "text": "Why is the sky blue short answer?<|>What is the true color of our sky?<|>How to explain to a kid why the sky is blue?<|>Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar", - "title": null, - "type": "people_also_ask", - "url": null - }, - { - "cite": "https://spaceplace.nasa.gov › blue-sky", - "cmpt_rank": 2, - "details": null, - "error": null, - "section": "main", - "serp_rank": 2, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://spaceplace.nasa.gov/blue-sky/" - }, - { - "cite": "160+ comments · 2 years ago", - "cmpt_rank": 3, - "details": { - "items": [ - { - "text": "Eli5: why is the sky blue? : r/explainlikeimfive - Reddit", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1iytxes/eli5_why_is_the_sky_blue/" - }, - { - "text": "I know why the sky is blue but why is the light that ...", - "url": "https://www.reddit.com/r/askscience/comments/1vcwur/i_know_why_the_sky_is_blue_but_why_is_the_light/" - }, - { - "text": "More results from www.reddit.com", - "url": "/search?q=why+is+the+sky+blue?+site:www.reddit.com&sca_esv=bd16726c6a722966&sa=X&ved=2ahUKEwi1uKmwmsSSAxW2QjABHazhBQcQrQIoAnoECB4QAw" - } - ], - "type": "hyperlinks" - }, - "error": null, - "section": "main", - "serp_rank": 3, - "sub_rank": 0, - "sub_type": "submenu", - "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", - "title": "Why is the sky blue? Do I understand it correctly", - "type": "general", - "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" - }, - { - "cite": "https://kids.nationalgeographic.com › books › article", - "cmpt_rank": 4, - "details": null, - "error": null, - "section": "main", - "serp_rank": 4, - "sub_rank": 0, - "sub_type": null, - "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today.", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://kids.nationalgeographic.com/books/article/sky" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 5, - "sub_rank": 0, - "sub_type": "what_people_are_saying", - "text": null, - "title": "We see the sky as blue, but in reality it is completely black. The blue color is simply an optical illusion caused by the scattering of sunlight in our atmosphere.", - "type": "perspectives", - "url": "https://www.facebook.com/groups/ScienceKiDuniya/posts/3440058569495988/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 6, - "sub_rank": 1, - "sub_type": "what_people_are_saying", - "text": null, - "title": "The REAL Reason the Sky Looks Blue", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 7, - "sub_rank": 2, - "sub_type": "what_people_are_saying", - "text": null, - "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", - "type": "perspectives", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 8, - "sub_rank": 3, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Sky Blue?", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/n4ElQOE3lyI" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 9, - "sub_rank": 4, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Sky Blue? Credit to Discover.", - "type": "perspectives", - "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 10, - "sub_rank": 5, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why Is the Sky Blue? Science Behind the Blue Sky\n\n#WhyIsTheSkyBlue\n#BlueSky\n#ScienceExplained\n#RayleighScattering\n#spacescience", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DT7ufKNj70z/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 11, - "sub_rank": 6, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Night Sky Black and not Blue? | February 2 - February 8 | Star Gazers", - "type": "perspectives", - "url": "https://www.youtube.com/watch?v=pIZBeINyK8w" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 12, - "sub_rank": 7, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 13, - "sub_rank": 8, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Did you know that the sky in GTA V is blue because it is a reference to the real life sky which is also blue", - "type": "perspectives", - "url": "https://www.reddit.com/r/GTAV/comments/1qtseyg/did_you_know_that_the_sky_in_gta_v_is_blue/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 14, - "sub_rank": 9, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", - "type": "perspectives", - "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 15, - "sub_rank": 10, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/o1pTCw8cg10" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 16, - "sub_rank": 11, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why are #sunsets red but the #sky is blue? They are both caused by the same thing. \n\nThe light from the sun is white-ish. That means it's a combination of red, green and blue light. It turns out that the blue light is scattered by the air (nitrogen and oxygen molecules). This scattered blue light enters your eyes and you see the sky as blue.\n\nFor the sunset, the sun is low in the sky and passes through MORE air and that means that MORE of the blue light has been scattered. With less blue light from the sun, it looks more red.\n\nHere's a great physics demo. Put some powdered coffee creamer in water and shine a flashlight through it. If you look at the water from the side, it looks blueish. Looking into the flashlight it looks more red. Just like the sunset and sky.\n\n#physics #science #stem #physicsdemo", - "type": "perspectives", - "url": "https://www.instagram.com/p/DR2aqixgJUa/" - }, - { - "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", - "cmpt_rank": 6, - "details": null, - "error": null, - "section": "main", - "serp_rank": 17, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time.", - "title": "Why Is the Sky Blue? - NESDIS - NOAA", - "type": "general", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" - }, - { - "cite": "https://math.ucr.edu › home › baez › physics › General", - "cmpt_rank": 7, - "details": null, - "error": null, - "section": "main", - "serp_rank": 18, - "sub_rank": 0, - "sub_type": null, - "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" - }, - { - "cite": "864K+ views · 5 years ago", - "cmpt_rank": 8, - "details": null, - "error": null, - "section": "main", - "serp_rank": 19, - "sub_rank": 0, - "sub_type": "video", - "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" - }, - { - "cite": "50+ reactions · 1 year ago", - "cmpt_rank": 9, - "details": null, - "error": null, - "section": "main", - "serp_rank": 20, - "sub_rank": 0, - "sub_type": "video", - "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", - "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", - "type": "general", - "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" - }, - { - "cite": "https://www.britannica.com › ... › Matter & Energy", - "cmpt_rank": 10, - "details": null, - "error": null, - "section": "main", - "serp_rank": 21, - "sub_rank": 0, - "sub_type": null, - "text": "Jan 22, 2026 — The midday sky appears blue, rather than a combination of blue and violet, because our eyes are more sensitive to blue light than to violet light. Read more", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://www.britannica.com/story/why-is-the-sky-blue" - }, - { - "cite": "https://www.uu.edu › dept › physics › scienceguys", - "cmpt_rank": 11, - "details": null, - "error": null, - "section": "main", - "serp_rank": 22, - "sub_rank": 0, - "sub_type": null, - "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", - "title": "Why is the sky blue on Earth, but black in space or ...", - "type": "general", - "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" - }, - { - "cite": null, - "cmpt_rank": 12, - "details": { - "heading": "People also search for", - "items": [ - "Why is the sky blue Class 10", - "Why is the sky blue for kids", - "Why is the sky blue no ai", - "Why is the sky blue reflection of ocean", - "Why is the sky blue and not violet", - "Why is the sky blue Army", - "Why is the sky blue long answer", - "Why is the sky blue essay" - ], - "type": "text" - }, - "error": null, - "section": "footer", - "serp_rank": 23, - "sub_rank": 0, - "sub_type": "people_also_search_for", - "text": "Why is the sky blue Class 10<|>Why is the sky blue for kids<|>Why is the sky blue no ai<|>Why is the sky blue reflection of ocean<|>Why is the sky blue and not violet<|>Why is the sky blue Army<|>Why is the sky blue long answer<|>Why is the sky blue essay", - "title": null, - "type": "searches_related", - "url": null - } - ] -} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[052072e83c79].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[052072e83c79].json new file mode 100644 index 00000000..3596c3f7 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[052072e83c79].json @@ -0,0 +1,256 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": false, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 1920000.0, + "result_estimate_time": 0.23 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "unavailable", + "text": null, + "title": null, + "type": "ai_overview", + "url": null + }, + { + "cite": "https://www.tesla.com › secret-master-plan", + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Aug 2, 2006 — The Tesla Roadster is designed to beat a gasoline sports car like a Porsche or Ferrari in a head to head showdown.", + "title": "The Secret Tesla Motors Master Plan (just between you ...", + "type": "general", + "url": "https://www.tesla.com/secret-master-plan" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": { + "items": [ + "What is Tesla's mission statement in 2024?", + "What is the promise of the Tesla brand?", + "What is Tesla's vision statement?", + "What is the philosophy of Tesla?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "What is Tesla's mission statement in 2024?<|>What is the promise of the Tesla brand?<|>What is Tesla's vision statement?<|>What is the philosophy of Tesla?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.tesla.com › about", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "We're building a world powered by solar energy, running on batteries and transported by electric vehicles. Explore the most recent impact of our products.", + "title": "About", + "type": "general", + "url": "https://www.tesla.com/about" + }, + { + "cite": "https://teslauniverse.com › nikola-tesla › articles › strikin...", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "It is most sumptuously got up on vellum paper and altogether constitutes a manifesto worthy of the original genius issuing it. It is to be gathered from the ...", + "title": "A Striking Tesla Manifesto", + "type": "general", + "url": "https://teslauniverse.com/nikola-tesla/articles/striking-tesla-manifesto" + }, + { + "cite": "https://finance.yahoo.com › news › tesla-scrubs-elon-m...", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "Aug 28, 2024 — Tesla's “Secret Master Plan” outlined the company's aim to produce affordable EVs in line with its long-term sustainability goals.", + "title": "Tesla scrubs Elon Musk's original brainchild manifesto from ...", + "type": "general", + "url": "https://finance.yahoo.com/news/tesla-scrubs-elon-musk-original-210311141.html" + }, + { + "cite": "9.9K+ views · 4 weeks ago", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": "video", + "text": "The Las Vegas Metropolitan Police Department alongside the ATF provided an update into theTeslaCybertruck explosion on Tuesday.", + "title": "Tesla Cybertruck explosion: Police discover additional ...", + "type": "general", + "url": "https://www.youtube.com/watch?v=r-fgimurh2g" + }, + { + "cite": "https://www.tesla.com › master-plans", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "Tesla's mission is to accelerate the world's transition to sustainable energy. Read Tesla's master plans.", + "title": "Master Plans", + "type": "general", + "url": "https://www.tesla.com/master-plans" + }, + { + "cite": "https://www.ktnv.com › news › watch-new-details-relea...", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "Jan 7, 2025 — LVMPD Sheriff Kevin McMahill said they have recovered a six-page document on Livelsberger's phone that he describes as a manifesto .", + "title": "Investigators: Soldier used AI to plan explosion, new 6-page ...", + "type": "general", + "url": "https://www.ktnv.com/news/watch-new-details-released-in-cybertruck-explosion-that-killed-soldier-in-las-vegas" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": { + "heading": "About Tesla", + "img_url": null, + "text": "About Tesla|About Tesla|About Tesla|About Tesla|Tesla, Inc. is an American multinational automotive and clean energy company. Headquartered in Austin, Texas, it designs, manufactures and sells battery electric vehicles, stationary battery energy storage devices from home to grid-scale, solar panels and solar shingles, and related products and services.|Tesla, Inc. is an American multinational automotive and clean energy company. Headquartered in Austin, Texas, it designs, manufactures and sells battery electric vehicles, stationary battery energy storage devices from home to grid-scale, solar panels and solar shingles, and related products and services.|Tesla, Inc. is an American multinational automotive and clean energy company. Headquartered in Austin, Texas, it designs, manufactures and sells battery electric vehicles, stationary battery energy storage devices from home to grid-scale, solar panels and solar shingles, and related products and services.|From Wikipedia|From Wikipedia|Wikipedia|Wikipedia|We're building a world powered by solar energy, running on batteries and transported by electric vehicles. Explore the most recent impact of our products, people and supply chain. Explore Impact. We design sustainable systems that are massively scalable—resulting in the greatest environmental benefit possible.|We're building a world powered by solar energy, running on batteries and transported by electric vehicles. Explore the most recent impact of our products, people and supply chain. Explore Impact. We design sustainable systems that are massively scalable—resulting in the greatest environmental benefit possible.|We're building a world powered by solar energy, running on batteries and transported by electric vehicles. Explore the most recent impact of our products, people and supply chain. Explore Impact. We design sustainable systems that are massively scalable—resulting in the greatest environmental benefit possible.|In their own words|In their own words|www.tesla.com|www.tesla.com|Tesla | TSLA Stock Price, Company Overview & News|Tesla | TSLA Stock Price, Company Overview & News|Tesla, Inc. engages in the design, development, manufacture, and sale of fully electric vehicles, energy generation and storage systems.|Tesla, Inc. engages in the design, development, manufacture, and sale of fully electric vehicles, energy generation and storage systems.|Forbes|Forbes|More about Tesla", + "type": "panel", + "urls": [ + { + "text": "", + "url": "https://en.wikipedia.org/wiki/Tesla,_Inc." + }, + { + "text": "", + "url": "https://www.tesla.com/about" + }, + { + "text": "", + "url": "https://www.forbes.com/companies/tesla/" + }, + { + "text": "More about Tesla", + "url": "/search?q=About+tesla.com&tbm=ilp&ctx=ob&sa=X&ved=2ahUKEwiTqOnR-6-LAxV2ulYBHSBjCsgQ8YkKegQIShAI" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": "panel", + "text": null, + "title": null, + "type": "knowledge", + "url": null + }, + { + "cite": "https://www.forbes.com › Innovation › Transportation", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Aug 26, 2024 — An 18-year-old manifesto in which the tech entrepreneur laid out his vision for the electric car maker shortly after its first public event ...", + "title": "Elon Musk's 2006 Climate Manifesto For Tesla Is Gone ...", + "type": "general", + "url": "https://www.forbes.com/sites/alanohnsman/2024/08/26/elon-musk-climate-manifesto-vanishes-from-tesla-website/" + }, + { + "cite": "https://coloradotimesrecorder.com › 2025/01 › davis-w...", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "Rather, I suspect that much of the media has opted to ignore the manifestos for the opposite reason: because the ideas expressed in those ...", + "title": "DAVIS: Why the Media Mostly Ignored the Tesla Bomber's ...", + "type": "general", + "url": "https://coloradotimesrecorder.com/2025/01/davis-why-the-media-mostly-ignored-the-tesla-bombers-shockingly-mainstream-manifesto/66654/" + }, + { + "cite": "220+ reactions · 1 month ago", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "To kick off the new year, let's dive into one of Tesla's boldest declarations—his 4-page manifesto , issued on January 1, 1904—a powerful ...", + "title": "Nikola Tesla's 1904 Manifesto – A... - Dr. ...", + "type": "general", + "url": "https://www.facebook.com/AheadOfHisTime/posts/nikola-teslas-1904-manifesto-a-vision-for-the-future-happy-new-year-everyone-tha/1135860038548152/" + }, + { + "cite": null, + "cmpt_rank": 13, + "details": { + "heading": "People also search for", + "items": [ + "Tesla manifesto pdf", + "Tesla Master Plan 2", + "Tesla Master Plan 3", + "Tesla Master Plan 1 PDF", + "Tesla mission", + "Tesla business strategy PDF", + "Tesla Master Plan 4", + "Tesla history" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "people_also_search_for", + "text": "Tesla manifesto pdf<|>Tesla Master Plan 2<|>Tesla Master Plan 3<|>Tesla Master Plan 1 PDF<|>Tesla mission<|>Tesla business strategy PDF<|>Tesla Master Plan 4<|>Tesla history", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[054791556a3a].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[054791556a3a].json new file mode 100644 index 00000000..4a9a68d8 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[054791556a3a].json @@ -0,0 +1,362 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard-overview", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 119000000.0, + "result_estimate_time": 0.45 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": "Mater", + "img_url": null, + "text": "Mater|Film character|Film character|Overview|Overview|Overview|Movies|Movies|Movies|Videos|Videos|Videos|Played by|Played by|Played by|Quotes|Quotes|Quotes", + "type": "panel" + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "panel", + "text": null, + "title": "Mater Film character Overview Movies Videos Played by Quotes", + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "heading": null, + "img_url": null, + "type": "panel", + "urls": [ + { + "text": "", + "url": "/search?sca_esv=52b056e1a97f92b3&q=mater+cars&tbm=isch&source=univ&sa=X&ved=2ahUKEwiovbbmhPCEAxWYFVkFHb95DtkQnN8JegQIEBAD" + }, + { + "text": "6:55|YouTube| • |Disney Junior|Tokyo Mater | Pixar's Cars Toon - Mater’s Tall Tales | Episode ...|A routine tow lands Mater in Tokyo, where he is challenged to a drift-style race against a nefarious gang leader and his posse of ninjas.|2 weeks ago", + "url": "https://www.youtube.com/watch?v=5jMvKu4j3lA" + }, + { + "text": "", + "url": "https://www.youtube.com/watch?v=4VxEPhl03ww" + }, + { + "text": "YouTube| • |Pixar Cars|1:16:37|Nov 11, 2023|Best of Tow Mater's Funniest Moments | Compilation | Pixar Cars", + "url": "https://www.youtube.com/watch?v=8mN4loP4b3s" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "featured_results", + "text": "6:55 YouTube • Disney Junior Tokyo Mater | Pixar's Cars Toon - Mater’s Tall Tales | Episode ... A routine tow lands Mater in Tokyo, where he is challenged to a drift-style race against a nefarious gang leader and his posse of ninjas. 2 weeks ago YouTube Best of Tow Mater | Pixar Cars - YouTube Sir Tow Mater is the name! Taking a look back at Mater's best moments in Pixar Cars! From his adventures of tow trucking, saving the world ... Jun 7, 2023 YouTube • Pixar Cars 1:16:37 Nov 11, 2023 Best of Tow Mater's Funniest Moments | Compilation | Pixar Cars", + "title": null, + "type": "knowledge", + "url": "https://www.youtube.com/watch?v=5jMvKu4j3lA" + }, + { + "cite": "https://pixarcars.fandom.com › wiki › Mater", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Mater is one of the six playable characters, the other five being Lightning McQueen, Cruz Ramirez, Sally, Ramone, and Guido, who are available from the start in ...", + "title": "Mater | Pixar Cars Wiki - Fandom", + "type": "general", + "url": "https://pixarcars.fandom.com/wiki/Mater" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": { + "items": [ + "What kind of car is Mater?", + "Is Mater's girlfriend in Cars 3?", + "Is Mater rich in Cars?", + "What happened to Mater in Cars 3?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "What kind of car is Mater?<|>Is Mater's girlfriend in Cars 3?<|>Is Mater rich in Cars?<|>What happened to Mater in Cars 3?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://en.wikipedia.org › wiki › Tow_Mater", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Sir Tow Mater KG is an anthropomorphic tow truck of the Pixar Cars franchise. His appearances include the feature films Cars , Cars 2, and Cars 3, ...", + "title": "Tow Mater", + "type": "general", + "url": "https://en.wikipedia.org/wiki/Tow_Mater" + }, + { + "cite": "2M+ followers", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "Did you know Mater isn't just Lightning McQueen's best friend but also “The World's Best Backwards Driver?” Here are some of our favorite videos of Tow ...", + "title": "Best of Mater | Pixar Cars", + "type": "general", + "url": "https://www.youtube.com/playlist?list=PLjnjB1Z6t-kWlcoGCm4IOgVQSJwayjngQ" + }, + { + "cite": "YouTube", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": "unspecified-3", + "text": null, + "title": "Best of Tow Mater's Funniest Moments | Compilation | Pixar Cars YouTube · Pixar Cars 1 hour, 16 minutes, 37 seconds Nov 11, 2023", + "type": "videos", + "url": "https://www.youtube.com/watch?v=8mN4loP4b3s" + }, + { + "cite": "YouTube", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 1, + "sub_type": "unspecified-3", + "text": null, + "title": "Tokyo Mater | Pixar's Cars Toon - Mater’s Tall Tales | Episode ... YouTube · Disney Junior 6 minutes, 55 seconds 2 weeks ago", + "type": "videos", + "url": "https://www.youtube.com/watch?v=5jMvKu4j3lA" + }, + { + "cite": "YouTube", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 2, + "sub_type": "unspecified-3", + "text": null, + "title": "Best of Mater's Adventures | Pixar's Cars Toon - Mater’s Tall ... YouTube · Pixar Cars 7 minutes, 35 seconds Dec 27, 2023", + "type": "videos", + "url": "https://www.youtube.com/watch?v=RMmB8ss4BRI" + }, + { + "cite": "YouTube", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 3, + "sub_type": "unspecified-3", + "text": null, + "title": "Lightning McQueen and Mater Race in Tokyo | Pixar's Cars ... YouTube · Pixar Cars 1 minute, 55 seconds 6 days ago", + "type": "videos", + "url": "https://www.youtube.com/watch?v=S4GnwSo0s0A" + }, + { + "cite": "https://disney.fandom.com › wiki › Mater", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "He is a rusty old tow truck and Lightning McQueen's best friend. He is a major character in Cars , the protagonist of Cars 2, and a supporting character in Cars  ...", + "title": "Mater | Disney Wiki - Fandom", + "type": "general", + "url": "https://disney.fandom.com/wiki/Mater" + }, + { + "cite": null, + "cmpt_rank": 8, + "details": { + "heading": "Movies", + "img_url": null, + "items": [ + "Cars", + "Cars 2", + "Cars 3", + "Tokyo Mater", + "Mater and the Ghostlight", + "Cars Toon: Hiccups" + ], + "type": "panel", + "urls": [] + }, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": "movies", + "text": "Cars", + "title": "Movies", + "type": "knowledge", + "url": null + }, + { + "cite": "https://www.amazon.com › Toys-Games-Tow-Mater › r...", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "Mattel Disney and Pixar Cars Toys, Radiator Springs 3-Pack of Die-cast Toy Cars & Trucks with Lightning McQueen, Mater & Sheriff. Car .", + "title": "Tow Mater: Toys & Games", + "type": "general", + "url": "https://www.amazon.com/Toys-Games-Tow-Mater/s?rh=n%3A165793011%2Cp_lbr_characters_browse-bin%3ATow+Mater" + }, + { + "cite": "3.7M+ views · 9 months ago", + "cmpt_rank": 10, + "details": { + "duration": "17:52", + "source": null, + "type": "video" + }, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "video", + "text": "Sir TowMateris the name! Taking a look back atMater'sbest moments in PixarCars! From his adventures of tow trucking, saving the world ...", + "title": "Best of Tow Mater | Pixar Cars - YouTube", + "type": "general", + "url": "https://www.youtube.com/watch?v=4VxEPhl03ww" + }, + { + "cite": "1.3M+ views · 8 months ago", + "cmpt_rank": 11, + "details": { + "duration": "6:20", + "source": null, + "type": "video" + }, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": "video", + "text": "Go undercover withMaterin his secret spy missions fromCars2. Watch as he becomes one of the best undercover spies and saves Lightning ...", + "title": "Mater's Spy Moments in Cars 2 | Pixar Cars - YouTube", + "type": "general", + "url": "https://www.youtube.com/watch?v=dGXkG8caVyM" + }, + { + "cite": "https://www.disneystore.com › lightning-mcqueen-and-...", + "cmpt_rank": 12, + "details": { + "price": "ating: 3.7", + "stock": " 3 votes", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "Lightning McQueen and Tow Mater Die Cast Set – Cars on the Road. $24.99 $19.98. 3.7. (3).", + "title": "Lightning McQueen and Tow Mater Die Cast Set", + "type": "general", + "url": "https://www.disneystore.com/lightning-mcqueen-and-tow-mater-die-cast-set-cars-on-the-road-461024885989.html" + }, + { + "cite": "https://www.disneystore.asia › ... › Vehicles & RC Toys", + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 0, + "sub_type": null, + "text": "Product Details · Inspired by Disney Pixar Cars on the Road · Mater features printed details · Includes: 1 laser gun, 1 meteor hammer, 1 speaker, and 1 new body ...", + "title": "Mater Die-Cast, Disney Pixar Cars on the Road", + "type": "general", + "url": "https://www.disneystore.asia/mater-die-cast-disney-pixar-cars-on-the-road-461023737517.html" + }, + { + "cite": null, + "cmpt_rank": 14, + "details": { + "heading": "Related searches", + "items": [ + "Mater cars for sale", + "Mater cars song", + "Tow mater cars", + "mater cars toys", + "mater cars voice", + "how old is mater in cars", + "mater cars crocs", + "why is mater rusty" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "Mater cars for sale<|>Mater cars song<|>Tow mater cars<|>mater cars toys<|>mater cars voice<|>how old is mater in cars<|>mater cars crocs<|>why is mater rusty", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 15, + "details": { + "items": [ + "Car", + "Voiced by", + "Quotation" + ], + "type": "panel" + }, + "error": null, + "section": "rhs", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": "panel_rhs", + "text": "Sir Tow Mater KG is an anthropomorphic tow truck of the Pixar Cars franchise. His appearances include the feature films Cars, Cars 2, and Cars 3, as well as in the TV series Cars Toons and Cars on the Road. Mater is also a playable character in each of the Cars video game installments.", + "title": "Things to know", + "type": "knowledge", + "url": "https://en.wikipedia.org/wiki/Tow_Mater" + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[2464d0d52a14].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[2464d0d52a14].json new file mode 100644 index 00000000..0b2c855a --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[2464d0d52a14].json @@ -0,0 +1,419 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 6030000000.0, + "result_estimate_time": 0.36 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "header", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "location_choose_area", + "text": null, + "title": "Results for United States", + "type": "notice", + "url": null + }, + { + "cite": "https://www.movementgyms.com", + "cmpt_rank": 1, + "details": { + "items": [ + { + "title": "Membership & Classes", + "url": "https://movementgyms.com/memberships-passes/" + }, + { + "title": "Memberships", + "url": "/aclk?sa=l&ai=DChcSEwjU6ei0ivCEAxWHR0cBHW5AAFoYABACGgJxdQ&gclid=EAIaIQobChMI1OnotIrwhAMVh0dHAR1uQABaEAAYASACEgK17fD_BwE&sig=AOD64_3jtsyITkCu7ya3QW07WDwRX4b5pw&adurl=&q=" + }, + { + "title": "Yoga", + "url": "https://movementgyms.com/portland/yoga/" + }, + { + "title": "Gym Map & Location", + "url": "https://movementgyms.com/location-finder/" + }, + { + "title": "Climbing", + "url": "https://movementgyms.com/portland/climbing/" + }, + { + "title": "Fitness", + "url": "https://movementgyms.com/portland/fitness/" + } + ], + "type": "menu" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "submenu", + "text": "Join OR's Top Climbing Gym - Movement Gyms", + "title": "Join OR's Top Climbing Gym - Movement Gyms", + "type": "ad", + "url": "https://movementgyms.com/portland/" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": { + "heading": "Choose what you’re giving feedback on", + "img_url": null, + "text": "an act of changing physical location or position or of having this changed. \"a slight movement of the upper body\" | a change or development. \"movements in the underlying financial markets\"", + "type": "panel", + "urls": [ + { + "text": "Oxford Languages", + "url": "https://languages.oup.com/google-dictionary-en" + }, + { + "text": "Learn more", + "url": "https://support.google.com/websearch/answer/10106608?hl=en" + }, + { + "text": " | |Learn to pronounce| | ", + "url": "/search?sca_esv=52b056e1a97f92b3&q=how+to+pronounce+movement&stick=H4sIAAAAAAAAAOMIfcRoxS3w8sc9YSnDSWtOXmPU5uINKMrPK81LzkwsyczPExLhYglJLcoV4pHi4uLIzS9LzU3NK7FiUWJKzeNZxCqZkV-uUJKvUADUkw_UlKoAUwIAXA9cGFsAAAA&pron_lang=en&pron_country=us&sa=X&ved=2ahUKEwjj4960ivCEAxWpK1kFHQKJCZUQ3eEDegQIJhAM" + }, + { + "text": "See more", + "url": "/search?sca_esv=52b056e1a97f92b3&q=movement&si=AKbGX_q870E3DK3nJ7cu3BOD7pxC_aqMorQHhVF1baCEVu3Vd_V5_p4m9t8XdovWuzAVjFv3iWnro3sYmZj5OLbuIEMj6VfJyY4p3-FnMXL5CJ_85bA3438%3D&expnd=1&sa=X&ved=2ahUKEwjj4960ivCEAxWpK1kFHQKJCZUQ2v4IegUIJhCdAQ" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": "dictionary", + "text": "an act of changing physical location or position or of having this changed. \"a slight movement of the upper body\" | a change or development. \"movements in the underlying financial markets\"", + "title": "movement", + "type": "knowledge", + "url": null + }, + { + "cite": "https://movementgyms.com", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Movement looks to transform both people and the planet through climbing, yoga, and fitness - a giving community that brings out the best in its members.", + "title": "Movement Climbing, Yoga & Fitness | Climb As You Are", + "type": "general", + "url": "https://movementgyms.com/" + }, + { + "cite": "https://www.merriam-webster.com › dictionary › move...", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "2 days ago — The meaning of MOVEMENT is the act or process of moving; especially : change of place or position or posture. How to use movement in a ...", + "title": "Movement Definition & Meaning", + "type": "general", + "url": "https://www.merriam-webster.com/dictionary/movement" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": { + "items": [ + "What is the definition of a movement?", + "What is a word that means movement?", + "What is an example of a movement?", + "How do you describe movement?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "What is the definition of a movement?<|>What is a word that means movement?<|>What is an example of a movement?<|>How do you describe movement?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://movementfestival.com", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "Movement Music Festival is one of the longest-running dance music events in the world, committed to showcasing authentic electronic music.", + "title": "Movement Music Festival - Detroit", + "type": "general", + "url": "https://movementfestival.com/" + }, + { + "cite": "https://movement.com", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "Get approved. Move home. Love people. Experience loan and refinance process designed for you — with profits designed for others.", + "title": "Movement Mortgage - Home Loans and Refinance", + "type": "general", + "url": "https://movement.com/" + }, + { + "cite": "https://movementgyms.com › boulder", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "Movement is located in the heart of Boulder, a city known for the earliest recorded climbing in the state way back in the early 1900's.", + "title": "Boulder | Movement Climbing, Yoga & Fitness", + "type": "general", + "url": "https://movementgyms.com/boulder/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": { + "category": "Physical fitness program", + "hours": "Open ⋅ Closes 8 PM", + "n_reviews": 98, + "phone": "(360) 441-5092", + "rating": 5.0, + "review_snippet": "I have experienced movement and training in my life, almost ...", + "type": "place", + "website": "https://www.evolvemoveplay.com/", + "years_in_business": "5+ years in business" + }, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Evolve Move Play<|>5.0<|>(98)<|> · Physical fitness program<|>5+ years in business · (360) 441-5092<|>Open ⋅ Closes 8 PM<|>\"I have experienced <|>movement<|> and training in my life, almost ...\"", + "title": "Evolve Move Play", + "type": "local_results", + "url": "https://www.evolvemoveplay.com/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": { + "address": "Dallas, TX", + "category": "Gym", + "directions": "/maps/dir//Movement+Design+District,+141+Glass+St,+Dallas,+TX+75207/data=!4m6!4m5!1m1!4e2!1m2!1m1!1s0x864e997833e8e8e3:0xc7a41fa9db91416b?sa=X&ved=1t:57443&ictx=111", + "hours": "Open ⋅ Closes 11 PM", + "n_reviews": 78, + "phone": "(469) 941-5100", + "rating": 4.7, + "review_snippet": "Great top ropes, great bouldering, great gym, great staff.", + "type": "place", + "website": "https://movementgyms.com/design-district/?utm_source=google-my-business&utm_medium=organic&utm_campaign=design-district" + }, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 1, + "sub_type": null, + "text": "Movement Design District<|>4.7<|>(78)<|> · Gym<|>Dallas, TX · (469) 941-5100<|>Open ⋅ Closes 11 PM<|>\"Great top ropes, great bouldering, great gym, great staff.\"", + "title": "Movement Design District", + "type": "local_results", + "url": "https://movementgyms.com/design-district/?utm_source=google-my-business&utm_medium=organic&utm_campaign=design-district" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": { + "address": "Plano, TX", + "category": "Gym", + "directions": "/maps/dir//Movement+Plano,+525+Talbert+Dr,+Plano,+TX+75093/data=!4m6!4m5!1m1!4e2!1m2!1m1!1s0x864c23fc79f3840d:0x56bac58d1ac2683f?sa=X&ved=1t:57443&ictx=111", + "hours": "Open ⋅ Closes 10 PM", + "n_reviews": 439, + "phone": "(972) 599-0865", + "rating": 4.4, + "review_snippet": "Willing to change review if I see change in their pricing.", + "type": "place", + "website": "https://movementgyms.com/plano/?utm_source=google-my-business&utm_medium=organic&utm_campaign=plano", + "years_in_business": "3+ years in business" + }, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 2, + "sub_type": null, + "text": "Movement Plano<|>4.4<|>(439)<|> · Gym<|>3+ years in business · Plano, TX · (972) 599-0865<|>Open ⋅ Closes 10 PM<|>\"Willing to change review if I see change in their pricing.\"", + "title": "Movement Plano", + "type": "local_results", + "url": "https://movementgyms.com/plano/?utm_source=google-my-business&utm_medium=organic&utm_campaign=plano" + }, + { + "cite": "https://www.dictionary.com › browse › movement", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "the course, tendency, or trend of affairs in a particular field. a diffusely organized or heterogeneous group of people or organizations tending toward or ...", + "title": "MOVEMENT Definition & Usage Examples", + "type": "general", + "url": "https://www.dictionary.com/browse/movement" + }, + { + "cite": "https://dictionary.cambridge.org › dictionary › movement", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": null, + "text": "MOVEMENT meaning: 1. a change of position: 2. what someone is doing during a particular period: 3. a group of…. Learn more.", + "title": "MOVEMENT | definition in the Cambridge English Dictionary", + "type": "general", + "url": "https://dictionary.cambridge.org/us/dictionary/english/movement" + }, + { + "cite": "https://movement.com › paymymortgage", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": null, + "text": "Locate your newly funded loan with Movement for the first time. Find My Loan. Red phone inside circle with a check. WANT TO PAY OFF YOUR LOAN?", + "title": "Pay My Mortgage", + "type": "general", + "url": "https://movement.com/paymymortgage" + }, + { + "cite": "https://twitter.com/movementlabsxyz", + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": "header", + "text": null, + "title": "Movement (@movementlabsxyz) · X X (Twitter) https://twitter.com/movementlabsxyz", + "type": "twitter_cards", + "url": "https://twitter.com/movementlabsxyz?ref_src=twsrc^google|twcamp^serp|twgr^author" + }, + { + "cite": "Posted on X · 23 hours ago", + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 1, + "sub_type": "card", + "text": "⚡ Resources in #Move 💎\n\nTransforming Digital Asset Management!\n\n✅ Single, Secure Location\n✅ Immutable Integrity\n✅ Linear Ownership\n\nSay no to:\n❌ Duplication Errors\n❌ Unauthorized Access\n❌ Immutable Tampering\n\nLet's dive into the heart of secure asset handling 👇 ", + "title": "@movementlabsxyz", + "type": "twitter_cards", + "url": "https://twitter.com/movementlabsxyz/status/1767378019415674922?ref_src=twsrc^google|twcamp^serp|twgr^tweet" + }, + { + "cite": "Posted on X · 2 days ago", + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 2, + "sub_type": "card", + "text": "The Movement Community is the best!\n\nWe asked for stickers and emojis.\n\nAs always, y'all went above and beyond 👏\n\nAnd... the Movers are voting on the best Moveus Memes right now in the Discord.\n\nCome join us! ⚡️ ", + "title": "@movementlabsxyz", + "type": "twitter_cards", + "url": "https://twitter.com/movementlabsxyz/status/1766976691354968379?ref_src=twsrc^google|twcamp^serp|twgr^tweet" + }, + { + "cite": "Posted on X · 4 days ago", + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 3, + "sub_type": "card", + "text": "We're collaborating with @Lyncworld to enable devs to build games and dApps with LYNC modular infra on M2 in just minutes!\n\nLYNC is a leading multi-chain gaming ecosystem providing key infra to Tier 1 studios & indie devs.\n\nAnd with M2 they're bringing #Move GameFi to Ethereum ⚡️ ", + "title": "@movementlabsxyz", + "type": "twitter_cards", + "url": "https://twitter.com/movementlabsxyz/status/1766165501011808625?ref_src=twsrc^google|twcamp^serp|twgr^tweet" + }, + { + "cite": "33.1M+ views · 5 years ago", + "cmpt_rank": 14, + "details": { + "duration": "3:56", + "source": null, + "type": "video" + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": "video", + "text": "Listen to 'Movement' here: https://Hozier.lnk.to/MovementIN Listen to the Be (Acoustic) here: https://Hozier.lnk.to/BeListenID Shop the 5th ...", + "title": "Hozier - Movement - YouTube", + "type": "general", + "url": "https://www.youtube.com/watch?v=OSye8OO5TkM" + }, + { + "cite": null, + "cmpt_rank": 15, + "details": { + "heading": "Related searches", + "items": [ + "movement mortgage", + "movement in hindi", + "movement in biology", + "movement definition in history", + "movement 2024", + "movement gym", + "movement climbing", + "movement synonym" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "movement mortgage<|>movement in hindi<|>movement in biology<|>movement definition in history<|>movement 2024<|>movement gym<|>movement climbing<|>movement synonym", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[2cb96b64e45d].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[2cb96b64e45d].json new file mode 100644 index 00000000..d72b204e --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[2cb96b64e45d].json @@ -0,0 +1,1249 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 709000000.0, + "result_estimate_time": 0.44 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "items": [ + { + "text": "With LCD Screen", + "url": "/search?q=drawing+tablet+with+lcd+screen&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyHwgBEg9XaXRoIExDRCBTY3JlZW4YAjoICJbjLBCX4yxYurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAB6BAgPEAM" + }, + { + "text": "Wacom", + "url": "/search?q=wacom+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyEwgBEgVXYWNvbToICIPzPBDc9z5YurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAF6BAgPEAQ" + }, + { + "text": "Huion", + "url": "/search?q=huion+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyEwgBEgVIdWlvbjoICIPzPBD39T5YurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAJ6BAgPEAU" + }, + { + "text": "Artist", + "url": "/search?q=artist+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFAgBEgZBcnRpc3Q6CAit2iwQr9osWLqxIGAB&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAN6BAgPEAY" + }, + { + "text": "Under $150", + "url": "/search?q=drawing+tablet+under+$150&shoprs=CAESDRILEQAAAACj4aFBGAEYBSoOZHJhd2luZyB0YWJsZXQyIwgFEgpVbmRlciAkMTUwGAIiDRILEQAAAACj4aFBGAEqAhgBWLqxIGAB&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAR6BAgPEAc" + }, + { + "text": "Mac OS", + "url": "/search?q=mac+os+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFggBEgZNYWMgT1M6CgiIkPQBEIqQ9AFYurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAV6BAgPEAg" + }, + { + "text": "Samsung", + "url": "/search?q=samsung+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFQgBEgdTYW1zdW5nOggIg_M8EO7mPVi6sSBgAQ&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAZ6BAgPEAk" + }, + { + "text": "Apple", + "url": "/search?q=apple+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyEwgBEgVBcHBsZToICIPzPBCZ3j1YurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAd6BAgPEAo" + }, + { + "text": "Get it fast", + "url": "/search?q=drawing+tablet&shoprs=CAESC5oBCAoGCAM4AUgBGBQqDmRyYXdpbmcgdGFibGV0MiIIFBILR2V0IGl0IGZhc3QiC5oBCAoGCAM4AUgBKgQQARgBYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAh6BAgPEAs" + }, + { + "text": "Windows OS", + "url": "/search?q=windows+os+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyGggBEgpXaW5kb3dzIE9TOgoIiJD0ARCJkPQBWLqxIGAB&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAl6BAgPEAw" + }, + { + "text": "Wireless", + "url": "/search?q=wireless+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFggBEghXaXJlbGVzczoICNvcLBDd3CxYurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAp6BAgPEA0" + }, + { + "text": "Microsoft", + "url": "/search?q=microsoft+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFwgBEglNaWNyb3NvZnQ6CAiD8zwQjuo9WLqxIGAB&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAt6BAgPEA4" + }, + { + "text": "Android", + "url": "/search?q=android+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFwgBEgdBbmRyb2lkOgoIiJD0ARCOkPQBWLqxIGAB&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKAx6BAgPEA8" + }, + { + "text": "On sale", + "url": "/search?q=drawing+tablet+sale&shoprs=CAESBEoCGAEYBioOZHJhd2luZyB0YWJsZXQyEwgGEgdPbiBzYWxlGAIiBEoCGAFYurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKA16BAgPEBA" + }, + { + "text": "Used", + "url": "/search?q=used+drawing+tablet&shoprs=CAEYCioOZHJhd2luZyB0YWJsZXQyCAgKEgRVc2VkWLqxIGAB&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKA56BAgPEBE" + }, + { + "text": "White", + "url": "/search?q=white+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFQgBEgVXaGl0ZToKCP3WowEQhdejAVi6sSBgAQ&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKA96BAgPEBI" + }, + { + "text": "Bluetooth", + "url": "/search?q=bluetooth+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFwgBEglCbHVldG9vdGg6CAjb3CwQ4O06WLqxIGAB&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKBB6BAgPEBM" + }, + { + "text": "Black", + "url": "/search?q=black+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFQgBEgVCbGFjazoKCP3WowEQ_tajAVi6sSBgAQ&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKBF6BAgPEBQ" + }, + { + "text": "Pink", + "url": "/search?q=pink+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFAgBEgRQaW5rOgoI_dajARCB16MBWLqxIGAB&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKBJ6BAgPEBU" + }, + { + "text": "With Stand", + "url": "/search?q=drawing+tablet+with+stand&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyHAgBEgpXaXRoIFN0YW5kGAI6CgiRkPQBEJKQ9AFYurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKBN6BAgPEBY" + }, + { + "text": "Blue", + "url": "/search?q=blue+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFAgBEgRCbHVlOgoI_dajARD_1qMBWLqxIGAB&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKBR6BAgPEBc" + }, + { + "text": "Green", + "url": "/search?q=green+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFQgBEgVHcmVlbjoKCP3WowEQhNejAVi6sSBgAQ&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKBV6BAgPEBg" + }, + { + "text": "Chrome OS", + "url": "/search?q=chrome+os+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyGQgBEglDaHJvbWUgT1M6CgiIkPQBEI2Q9AFYurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKBZ6BAgPEBk" + }, + { + "text": "Linux", + "url": "/search?q=linux+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyFQgBEgVMaW51eDoKCIiQ9AEQjJD0AVi6sSBgAQ&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKBd6BAgPEBo" + }, + { + "text": "Wired", + "url": "/search?q=wired+drawing+tablet&shoprs=CAEYASoOZHJhd2luZyB0YWJsZXQyEwgBEgVXaXJlZDoICNvcLBDc3CxYurEgYAE&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQip4GKBh6BAgPEBs" + } + ], + "type": "hyperlinks" + }, + "error": null, + "section": "header", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": null, + "type": "top_image_carousel", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$39.95", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "product", + "text": null, + "title": "Wacom Intuos Small Graphics Drawing Tablet, includes Training & Software; 4 Customizable ExpressKeys Compatible With Chromebook Mac Android & for $39.95 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/Wacom-Drawing-Software-Included-CTL4100/dp/B079HL9YSF?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=ATVPDKIKX0DER" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$799.95", + "source": "Wacom eStore", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 1, + "sub_type": "product", + "text": null, + "title": "Wacom Cintiq 16 | Drawing Tablet With Screen for $799.95 from Wacom eStore", + "type": "shopping_ads", + "url": "http://estore.wacom.com/en-us/wacom-cintiq-16-dtk1660k0a.html" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$48.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 2, + "sub_type": "product", + "text": null, + "title": "XOPPOX Graphics Drawing Tablet 10 x 6 Inch Large Active Area with 8192 Levels Battery-Free Pen and 12 Hot Keys, Compatible with PC/Mac/Android OS for $48.99 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/XOPPOX-Graphics-Battery-Free-Compatible-Painting/dp/B08TC1N6JS?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A1WLMZ7OC94OZG" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "n_reviews": "147", + "price": "$277.49", + "source": "XP-Pen Official...", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 3, + "sub_type": "product", + "text": null, + "title": "XPPen Artist 15.6Pro drawing tablet provides a one-year warranty, and lifetime technical support. for $277.49 from XP-Pen Official Store | US", + "type": "shopping_ads", + "url": "https://www.xp-pen.com/store/buy/artist-15-6-pro.html" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$59.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 4, + "sub_type": "product", + "text": null, + "title": "HUION Inspiroy H1060P Graphics Drawing Tablet with 8192 Pressure Sensitivity Battery-Free Stylus and 12 Customized Hot Keys, 10 x 6.25 inches for $59.99 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/HUION-Inspiroy-Graphics-Battery-Free-Sensitivity/dp/B01FTE9HS2?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A30BRCK3LE6SB5" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$19.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 5, + "sub_type": "product", + "text": null, + "title": "Drawing Tablet XPPen G430S OSU, Graphic Drawing Tablet with 8192 Levels Pressure Battery-Free Stylus, 4 x 3 inch Ultrathin, for OSU Game, Online for $19.99 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/Ultrathin-Graphic-Digital-Drawing-Pressure/dp/B01BA6XQI0?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=AABH22FTN87KM" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$89.00", + "source": "gaomon.net", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 6, + "sub_type": "product", + "text": null, + "title": "GAOMON Refurbished PD1161 11.6 Inches Portable Drawing Tablet, US Plug for $89.00 from gaomon.net", + "type": "shopping_ads", + "url": "https://gaomon.net/products/refurb-pd1161-pen-display?currency=USD&variant=48145944903985&utm_medium=cpc&utm_source=google&utm_campaign=Google%20Shopping&stkn=c5a45b3900de" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$548.99", + "source": "Huion Official St...", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 7, + "sub_type": "product", + "text": null, + "title": "Huion Kamvas Drawing Tablet 22 plus with Free Gift for $548.99 from Huion Official Store", + "type": "shopping_ads", + "url": "https://store.huion.com/products/kamvas-22-series-with-huion-keydial-mini?vid=1180" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$164.95", + "source": "Wacom eStore", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 8, + "sub_type": "product", + "text": null, + "title": "Refurb Wacom One | Pen Display for $164.95 from Wacom eStore", + "type": "shopping_ads", + "url": "http://estore.wacom.com/en-us/refurb-wacom-one-13-pen-display-udtc133w0a.html" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$39.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 9, + "sub_type": "product", + "text": null, + "title": "VEIKK A30 V2 Drawing Tablet 10x6 Inch Graphics Tablet with Battery-Free Pen and 8192 Professional Levels Pressure for $39.99 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/VEIKK-Graphics-Drawing-Tablet-Battery-Free/dp/B07DQK7H15?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A2LCUUA9Q9HAX3" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$29.63", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 10, + "sub_type": "product", + "text": null, + "title": "HUION Inspiroy H640P Drawing Tablet with Battery-Free Stylus 8192 Pressure Sensitivity 6 Hot Keys, 6x4-inch Graphics Pen Tablet for Digital Art & for $29.63 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/Inspiroy-H640P-Graphics-Battery-Free-Sensitivity/dp/B075T6MTJX?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A30BRCK3LE6SB5" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$49.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 11, + "sub_type": "product", + "text": null, + "title": "Graphics Drawing Tablets, UGEE M708 10 x 6 inch Large Active Area Drawing Tablet with 8 Hot Keys 8192 Levels Pen Graphic Tablets for Computer for $49.99 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/Graphics-UGEE-Drawing-Graphic-Creation/dp/B06WLGJDX5?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A1KMSU2IVTRQKC" + }, + { + "cite": "Amazon.com", + "cmpt_rank": 2, + "details": { + "n_reviews": "1K", + "price": "$239.99", + "rating": "4.4", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Wacom One Pen Display.", + "type": "products", + "url": null + }, + { + "cite": "Walmart - VEIKK official store", + "cmpt_rank": 2, + "details": { + "n_reviews": "32", + "price": "$169.99", + "rating": "4.7", + "source": "Walmart - VEIKK official store", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "VEIKK VK1200 Drawing Tablet with Screen, 11.6 Inch Full-Laminated Drawing Monitor Pen and Tilt Function.", + "type": "products", + "url": null + }, + { + "cite": "XP-Pen Official Store | US", + "cmpt_rank": 2, + "details": { + "n_reviews": "58", + "price": "$47.99", + "rating": "4.6", + "source": "XP-Pen Official Store | US", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "XPPen Deco 01 V2 Animation Drawing Tablet Drawing Pen Tablet with 60 Degrees of Tilt Brush and Supports Android Devices.", + "type": "products", + "url": null + }, + { + "cite": "Best Buy", + "cmpt_rank": 2, + "details": { + "n_reviews": "2.7K", + "price": "$59.99", + "rating": "4.5", + "source": "Best Buy", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Wacom Intuos S.", + "type": "products", + "url": null + }, + { + "cite": "Wacom eStore", + "cmpt_rank": 2, + "details": { + "n_reviews": "993", + "price": "$799.95", + "rating": "4.7", + "source": "Wacom eStore", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Wacom Cintiq 16.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 2, + "details": { + "n_reviews": "1.4K", + "price": "$39.99", + "rating": "4.9", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Veikk A30 V2 Drawing Tablet 10x6 Inch Graphics Tablet with Battery-Free Pen and 8192 Professional Levels Pressure.", + "type": "products", + "url": null + }, + { + "cite": "Walmart - Shenzhen Huion Animation Technolog Co., LTD", + "cmpt_rank": 2, + "details": { + "n_reviews": "97", + "price": "$169.00", + "rating": "4.5", + "source": "Walmart - Shenzhen Huion Animation Technolog Co., LTD", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Huion Kamvas Pro 12.", + "type": "products", + "url": null + }, + { + "cite": "XP-Pen Official Store | US", + "cmpt_rank": 2, + "details": { + "n_reviews": "374", + "price": "$223.99", + "rating": "4.8", + "source": "XP-Pen Official Store | US", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "XP-Pen XPPen Artist 13.3 Pro Digital Pen Display Graphic Drawing Monitor Animation Art Drawing Tablet With 8192 Pen Pressure and 99% Adobe RGB.", + "type": "products", + "url": null + }, + { + "cite": "https://www.amazon.com › drawing-tablet › k=drawing...", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": "image_strip", + "text": "HUION Inspiroy H1060P Graphics Drawing Tablet with 8192 Pressure Sensitivity Battery-Free Stylus and 12 Customized Hot Keys, 10 x 6.25 inches Digital Art Tablet ...", + "title": "Amazon.com: Drawing Tablet", + "type": "general", + "url": "https://www.amazon.com/drawing-tablet/s?k=drawing+tablet" + }, + { + "cite": "https://estore.wacom.com › en-us › tablets", + "cmpt_rank": 4, + "details": { + "price": "ree delivery over $50", + "stock": " Free 30", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "One by Wacom. Sketch , draw and edit photos with a simply natural pen experience. It couldn't be easier. Battery Free icon.", + "title": "Tablets", + "type": "general", + "url": "https://estore.wacom.com/en-us/tablets.html" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": { + "items": [ + "Which tablet is best for drawing?", + "What is the best drawing tablet for beginners?", + "What is the difference between a graphic tablet and a drawing tablet?", + "Can you use a drawing tablet without a computer?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 0, + "sub_type": null, + "text": "Which tablet is best for drawing?<|>What is the best drawing tablet for beginners?<|>What is the difference between a graphic tablet and a drawing tablet?<|>Can you use a drawing tablet without a computer?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://store.huion.com", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 0, + "sub_type": null, + "text": "Shop at Huion for professional graphic tablets , pen displays, LED tracing boards and accessories to bring creative ideas to life.", + "title": "Huion Official Store: Drawing Tablets, Pen Tablets, Pen ...", + "type": "general", + "url": "https://store.huion.com/" + }, + { + "cite": "https://www.bestbuy.com › site › abcat0513006", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 0, + "sub_type": "image_strip", + "text": "With a digital drawing pad , you can combine the control of hand drawing with a creative array of digital tools to make stunning digital art.", + "title": "Drawing Tablets: Digital Art Tablets", + "type": "general", + "url": "https://www.bestbuy.com/site/computer-accessories/graphics-tablets-digital-pens/abcat0513006.c?id=abcat0513006" + }, + { + "cite": "https://www.nytimes.com › Electronics › Accessories", + "cmpt_rank": 8, + "details": { + "price": "30 to $300", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "Jan 17, 2024 — Drawing tablets are nearly indispensable for creating art on a PC or laptop, and models such as the Huion Inspiroy 2 M are great for ...", + "title": "The 4 Best Drawing Tablets of 2024 | Reviews by Wirecutter", + "type": "general", + "url": "https://www.nytimes.com/wirecutter/reviews/best-drawing-tablets/" + }, + { + "cite": "Amazon.com", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 0, + "sub_type": "medium", + "text": null, + "title": "Drawing Tablet with Screen, 15.6'' XOPPOX Graphics Drawing Monitor Pen Display with 1080P Full Laminated Screen,Tilt 8192 Levels Battery-Free ... Amazon.com", + "type": "images", + "url": "https://www.amazon.com/XOPPOX-Drawing-Graphics-Battery-Free-Compatible/dp/B09MQRNS2M" + }, + { + "cite": "Amazon", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 1, + "sub_type": "medium", + "text": null, + "title": "GAOMON PD156Pro - 15.6\" Drawing Tablet with Screen Full-Laminated 120% sRGB Pen Display with Dial Key & 10 Hotkeys, Battery-Free Stylus Art Tablet for ... Amazon", + "type": "images", + "url": "https://www.amazon.com/GAOMON-PD156PRO-Full-Laminated-Tilt-Support-Battery-Free/dp/B08FXD2V9F" + }, + { + "cite": "ZDNet", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 2, + "sub_type": "medium", + "text": null, + "title": "The best drawing tablets of 2024: Expert recommended | ZDNET ZDNet", + "type": "images", + "url": "https://www.zdnet.com/article/best-drawing-tablet/" + }, + { + "cite": null, + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": "The best drawing tablets to buy right now", + "type": "most_read_articles", + "url": "https://www.theverge.com/23923747/best-graphics-drawing-tablets" + }, + { + "cite": null, + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 1, + "sub_type": null, + "text": null, + "title": "The best drawing tablets", + "type": "most_read_articles", + "url": "https://www.creativebloq.com/features/best-drawing-tablet" + }, + { + "cite": null, + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 2, + "sub_type": null, + "text": null, + "title": "The 7 Best Drawing Tablets To Enhance Your Digital Art", + "type": "most_read_articles", + "url": "https://www.forbes.com/sites/forbes-personal-shopper/article/best-drawing-tablets/" + }, + { + "cite": "YouTube", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 33, + "sub_rank": 0, + "sub_type": "unspecified-3", + "text": null, + "title": "10 Best DRAWING TABLETS To Buy In 2024 YouTube · Aaron Rutten 28 minutes, 29 seconds Dec 8, 2023", + "type": "videos", + "url": "https://www.youtube.com/watch?v=kaUOuYUDi5M" + }, + { + "cite": "YouTube", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 34, + "sub_rank": 1, + "sub_type": "unspecified-3", + "text": null, + "title": "Best Drawing Tablets for Beginners - What I Recommend YouTube · Brad Colbow 11 minutes, 32 seconds May 8, 2023", + "type": "videos", + "url": "https://www.youtube.com/watch?v=Dh49NwMt1yw" + }, + { + "cite": "YouTube", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 35, + "sub_rank": 2, + "sub_type": "unspecified-3", + "text": null, + "title": "The Cheapest Drawing Tablet VS the Most Expensive YouTube · Brad Colbow 12 minutes, 36 seconds Oct 17, 2023", + "type": "videos", + "url": "https://www.youtube.com/watch?v=rgvM59DBJag" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 36, + "sub_rank": 0, + "sub_type": "shopping", + "text": "Shop deals", + "title": "Save with deals on apparel, electronics, and more", + "type": "promo", + "url": "/search?sca_esv=52b056e1a97f92b3&q=shop+deals&sa=X&ved=2ahUKEwjhzu3NiPCEAxWGEFkFHY8tDcMQ_hF6BAh5EAA" + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 37, + "sub_rank": 0, + "sub_type": null, + "text": "How much does a graphic tablet cost?", + "title": "price", + "type": "buying_guide", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 38, + "sub_rank": 1, + "sub_type": null, + "text": "Including Wacom, Huion, XP-Pen", + "title": "popular brands", + "type": "buying_guide", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 39, + "sub_rank": 2, + "sub_type": null, + "text": "How do I know what size graphic tablet to get?", + "title": "sizes", + "type": "buying_guide", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 40, + "sub_rank": 3, + "sub_type": null, + "text": "Including Mac OS, Windows OS, Android", + "title": "popular types", + "type": "buying_guide", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 41, + "sub_rank": 4, + "sub_type": null, + "text": "What resolution should a graphics tablet be?", + "title": "resolution", + "type": "buying_guide", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 42, + "sub_rank": 5, + "sub_type": null, + "text": "What is the best drawing display tablet?", + "title": "display", + "type": "buying_guide", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 43, + "sub_rank": 6, + "sub_type": null, + "text": "How long is the warranty on a Huion tablet?", + "title": "warranty", + "type": "buying_guide", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 44, + "sub_rank": 7, + "sub_type": null, + "text": "What graphics tablet do professionals use?", + "title": "specs", + "type": "buying_guide", + "url": null + }, + { + "cite": "https://www.zdnet.com › Tech › Computing › Tablets", + "cmpt_rank": 14, + "details": null, + "error": null, + "section": "main", + "serp_rank": 45, + "sub_rank": 0, + "sub_type": null, + "text": "4 days ago — Wacom is one of the industry leaders when it comes to drawing and graphics tablets . With virtually no lag and 8,192 levels of pressure ...", + "title": "The best drawing tablets of 2024: Expert recommended | ZDNET", + "type": "general", + "url": "https://www.zdnet.com/article/best-drawing-tablet/" + }, + { + "cite": "https://www.theverge.com › best-graphics-drawing-tablets", + "cmpt_rank": 14, + "details": null, + "error": null, + "section": "main", + "serp_rank": 46, + "sub_rank": 1, + "sub_type": null, + "text": "Nov 2, 2023 — Wacom Cintiq Pro 27. $3500. The Wacom Cintiq Pro 27 is the gold standard for display tablets used by professional artists, with color accuracy ...", + "title": "The best drawing tablets to buy right now - The Verge", + "type": "general", + "url": "https://www.theverge.com/23923747/best-graphics-drawing-tablets" + }, + { + "cite": "https://www.parblo.com › collections › graphic-tablet", + "cmpt_rank": 15, + "details": null, + "error": null, + "section": "main", + "serp_rank": 47, + "sub_rank": 0, + "sub_type": "image_strip", + "text": "Picasso has been creating almost all his life, recording and expressing with drawings . Established in 2015, Parblo has been deeply involved ...", + "title": "Graphic Tablet", + "type": "general", + "url": "https://www.parblo.com/collections/graphic-tablet" + }, + { + "cite": "Best Buy", + "cmpt_rank": 16, + "details": { + "n_reviews": "2", + "price": "$99.95", + "rating": "4.5", + "source": "Best Buy", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 48, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Wacom One Graphics Tablet.", + "type": "products", + "url": null + }, + { + "cite": "Wacom eStore", + "cmpt_rank": 16, + "details": { + "n_reviews": "2.2K", + "price": "$299.95", + "rating": "4.6", + "source": "Wacom eStore", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 49, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Wacom Intuos Pro Large.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 16, + "details": { + "n_reviews": "620", + "price": "$39.99", + "rating": "4.9", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 50, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "VEIKK A30 V2 Drawing Tablet 10x6 Inch Graphics Tablet with Battery-free pen and 8192 Professional Levels Pressure. Low price.", + "type": "products", + "url": null + }, + { + "cite": "Walmart", + "cmpt_rank": 16, + "details": { + "n_reviews": "2.6K", + "price": "$39.95", + "rating": "4.6", + "source": "Walmart", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 51, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Wacom Intuos Pen Small. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Huion Official Store", + "cmpt_rank": 16, + "details": { + "n_reviews": "219", + "price": "$299.00", + "rating": "4.7", + "source": "Huion Official Store", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 52, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Huion KAMVAS Pro 16. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Best Buy", + "cmpt_rank": 16, + "details": { + "n_reviews": "33K", + "price": "$449.00", + "rating": "4.0", + "source": "Best Buy", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 53, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "reMarkable 2.", + "type": "products", + "url": null + }, + { + "cite": "Wacom eStore", + "cmpt_rank": 16, + "details": { + "n_reviews": "17", + "price": "$3,499.95", + "rating": "4.8", + "source": "Wacom eStore", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 54, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Wacom Cintiq Pro 27 Pen Display.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 16, + "details": { + "n_reviews": "49", + "price": "$18.98", + "rating": "4.8", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 55, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Orsen 10 Inch LCD Doodle Board Writing Tablet Colorful Drawing.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 17, + "details": { + "n_reviews": "5", + "price": "$39.99", + "rating": "4.6", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 56, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "XP-Pen Star G640.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 17, + "details": { + "n_reviews": "37", + "price": "$219.95", + "rating": "4.7", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 57, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "PicassoTab XL 11.6 inch Large Drawing Tablet | Simbans.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 17, + "details": { + "n_reviews": "80", + "price": "$1,249.95", + "rating": "4.8", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 58, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Wacom Cintiq 22.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 17, + "details": { + "n_reviews": "118", + "price": "$19.99", + "rating": "4.8", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 59, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "COOCOPCNT Xppen g430s 4x3 digital Drawing Tablet 8192 level OSU game.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 17, + "details": { + "n_reviews": "29", + "price": "$28.99", + "rating": "4.3", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 60, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "UGEE Drawing Tablet.", + "type": "products", + "url": null + }, + { + "cite": "Wacom eStore", + "cmpt_rank": 17, + "details": { + "n_reviews": "1.9K", + "price": "$59.95", + "rating": "4.5", + "source": "Wacom eStore", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 61, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Wacom Intuos Small Graphics Drawing Tablet. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Huion Official Store", + "cmpt_rank": 17, + "details": { + "n_reviews": "26", + "price": "$399.00", + "rating": "4.9", + "source": "Huion Official Store", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 62, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Huion Kamvas Drawing Tablet Kamvas 22 with 120% sRGB Color Gamut. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Wacom eStore", + "cmpt_rank": 17, + "details": { + "n_reviews": "1.5K", + "price": "$119.95", + "rating": "4.5", + "source": "Wacom eStore", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 63, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Wacom Intuos Wireless Graphics Drawing Tablet. SALE.", + "type": "products", + "url": null + }, + { + "cite": null, + "cmpt_rank": 18, + "details": { + "heading": "Related searches", + "items": [ + "drawing tablet with screen", + "drawing tablet for pc", + "wacom drawing tablet", + "drawing tablet for kids", + "best drawing tablet", + "drawing tablet for beginners" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 64, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "drawing tablet with screen<|>drawing tablet for pc<|>wacom drawing tablet<|>drawing tablet for kids<|>best drawing tablet<|>drawing tablet for beginners", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[2d6557bbff20].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[2d6557bbff20].json new file mode 100644 index 00000000..8ef70752 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[2d6557bbff20].json @@ -0,0 +1,285 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": false, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 8760000.0, + "result_estimate_time": 0.24 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "sections": [ + { + "heading": "Lack of the original manuscript", + "text": "Translators don't know which of the many copies is the correct one. Differences between copies are more significant in the New Testament." + }, + { + "heading": "Differences between manuscripts", + "text": "Scholars use the ecletic approach to combine manuscripts from different sources, but this approach relies on human judgment. Translators may choose a reading that's not in the standard printed texts." + }, + { + "heading": "Updating the language", + "text": null + } + ], + "sources": [ + { + "favicon": null, + "publisher": "Ken Collins", + "snippet": "Inclusive Language Can Be Self-Defeating in Bible Translations. It is necessary to be inclusive in contemporary speech and writing...", + "source_id": null, + "title": "The Problem with Inclusive-Language Bible Translations - Ken Collins", + "url": "https://www.kencollins.com/bible/bible-t6.htm#:~:text=Inclusive%20Language%20Can%20Be%20Self,preachers%20can%20interpret%20the%20text." + }, + { + "favicon": null, + "publisher": "Translation Journal", + "snippet": "Jul 18, 2018 — One basic problem inherent in Bible translation is that we do not have the original manuscript of the Bible, but copie...", + "source_id": null, + "title": "Problems of Bible Translation", + "url": "https://translationjournal.net/journal/18bible.htm#:~:text=in%20the%20future.-,One%20basic%20problem%20inherent%20in%20Bible%20translation%20is%20that%20we,is%20that%20of%20Ivan%20Panin." + }, + { + "favicon": null, + "publisher": "Blue Letter Bible", + "snippet": "BHS and the Nestle-Aland text are the standard Hebrew and Greek printed texts. While most modern translations use these printed te...", + "source_id": null, + "title": "What Are Some of the Key Issues Involved with Translating the Bible?", + "url": "https://www.blueletterbible.org/Comm/stewart_don/faq/bible-translations/question3-key-issues-with-translating-the-bible.cfm#:~:text=BHS%20and%20the%20Nestle%2DAland,1%20Samuel%2010:27%20NRSV)" + } + ], + "type": "ai_overview" + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "sectioned", + "text": "Bible translations can have problems due to a lack of the original manuscript, differences between manuscripts, and attempts to update the language .", + "title": null, + "type": "ai_overview", + "url": null + }, + { + "cite": "https://www.blueletterbible.org › faq › bible-translations", + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "There is also the issue of which theory of Bible translation is to be used. Should the translation be literal ? Should the work be more thought for thought?", + "title": "What Are Some of the Key Issues Involved with Translating ...", + "type": "general", + "url": "https://www.blueletterbible.org/Comm/stewart_don/faq/bible-translations/question3-key-issues-with-translating-the-bible.cfm" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": { + "items": [ + "What are the problems of Bible translation?", + "What is the most controversial translation of the Bible?", + "What is the most accurate translation of the Bible to the original?", + "Which translation of the Bible is closest to the original text?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "What are the problems of Bible translation?<|>What is the most controversial translation of the Bible?<|>What is the most accurate translation of the Bible to the original?<|>Which translation of the Bible is closest to the original text?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "50+ comments · 9 months ago", + "cmpt_rank": 3, + "details": { + "items": [ + { + "text": "What's the worst translation you've read of the Bible? - Reddit", + "url": "https://www.reddit.com/r/Christianity/comments/14mgmr3/whats_the_worst_translation_youve_read_of_the/" + }, + { + "text": "Theological bias in Bible translations. Looking for an ... - Reddit", + "url": "https://www.reddit.com/r/AcademicBiblical/comments/ru782y/theological_bias_in_bible_translations_looking/" + }, + { + "text": "Problems with bible translations : r/AcademicBiblical - Reddit", + "url": "https://www.reddit.com/r/AcademicBiblical/comments/dsx015/problems_with_bible_translations/" + }, + { + "text": "If the Bible was purposely translated incorrectly, why hasn't ...", + "url": "https://www.reddit.com/r/NoStupidQuestions/comments/linhyg/if_the_bible_was_purposely_translated_incorrectly/" + }, + { + "text": "More results from www.reddit.com", + "url": "/search?q=problems+with+bible+translations+site:www.reddit.com&sca_esv=14324ad49b0ab5cb&sa=X&ved=2ahUKEwis7bzp-6-LAxWArVYBHRbxNK4QrQIoBHoECB8QBQ" + } + ], + "type": "hyperlinks" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": "submenu", + "text": "As others have said, yes there are some issues with translation and copying errors . Some are accidental, some are deliberate, but it's not ...", + "title": "Has The Bible suffered from translation errors over ...", + "type": "general", + "url": "https://www.reddit.com/r/asklinguistics/comments/1clv3mq/has_the_bible_suffered_from_translation_errors/" + }, + { + "cite": "https://www.kencollins.com › bible › bible-t6", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Inclusive language translations put too much stress on what the reader does not want to hear , and too little stress on what the writer wants to say. Sometimes ...", + "title": "The Problem with Inclusive-Language Bible Translations", + "type": "general", + "url": "https://www.kencollins.com/bible/bible-t6.htm" + }, + { + "cite": "https://translationjournal.net › journal › 18bible", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "Jul 18, 2018 — One basic problem inherent in Bible translation is that we do not have the original manuscript of the Bible , but copies of copies of copies...", + "title": "Problems of Bible Translation", + "type": "general", + "url": "https://translationjournal.net/journal/18bible.htm" + }, + { + "cite": "https://textandcanon.org › what-makes-a-bible-translati...", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "Jul 11, 2023 — No translation is perfect. But really bad translations are idiosyncratic and mislead innocent Bible readers .", + "title": "What Makes a Bible Translation Really Bad?", + "type": "general", + "url": "https://textandcanon.org/what-makes-a-bible-translation-really-bad/" + }, + { + "cite": "https://www.critlarge.com › articles › 5-bible-translation...", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "Aug 19, 2019 — The following translations listed below should be avoided altogether, as utilizing them can lead to grave theological errors , the teaching ...", + "title": "5 Bible Translations You Should Avoid", + "type": "general", + "url": "https://www.critlarge.com/articles/2019/8/18/5-bible-translations-you-should-avoid" + }, + { + "cite": "https://www.quora.com › What-are-some-problems-wit...", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "The original one to two year-old lingo can be lost . In addition, there is the problem of Original meaning lost. God is who god is spelled out in ...", + "title": "What are some problems with translating the Bible into ...", + "type": "general", + "url": "https://www.quora.com/What-are-some-problems-with-translating-the-Bible-into-different-languages" + }, + { + "cite": "https://beinnotof.com › post › bibletranslations", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Apr 12, 2023 — In this article, I'll be dissecting everything about the different Bible translations, when to use certain ones, and overall, how to choose the right Bible  ...", + "title": "The Pros and Cons of Popular Bible Translations: A Guide ...", + "type": "general", + "url": "https://beinnotof.com/post/bibletranslations" + }, + { + "cite": "https://www.quora.com › What-are-some-of-the-proble...", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "The main issues with translation is that an entirely new language is created every time the word is created afresh . · Using a human ...", + "title": "What are some of the problems with modern Bible ...", + "type": "general", + "url": "https://www.quora.com/What-are-some-of-the-problems-with-modern-Bible-translations" + }, + { + "cite": "https://christianity.stackexchange.com › questions › wh...", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "Oct 29, 2017 — The translation of the NIV makes Christ appear to be without authority , and contradicts other scripture that affirms His nature and deity as the ...", + "title": "bible translation - What are the major criticisms of the NIV?", + "type": "general", + "url": "https://christianity.stackexchange.com/questions/60496/what-are-the-major-criticisms-of-the-niv" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": { + "heading": "People also search for", + "items": [ + "Best and worst Bible translations", + "Bible translations to avoid", + "How many translation errors are in the Bible", + "Inclusive language Bible online", + "Top 5 most accurate Bible translations", + "Bad Bible translations game", + "Best Bible translations", + "Why are there so many translations of the Bible" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": "people_also_search_for", + "text": "Best and worst Bible translations<|>Bible translations to avoid<|>How many translation errors are in the Bible<|>Inclusive language Bible online<|>Top 5 most accurate Bible translations<|>Bad Bible translations game<|>Best Bible translations<|>Why are there so many translations of the Bible", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[334a0bfcce0b].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[334a0bfcce0b].json new file mode 100644 index 00000000..8c7aab31 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[334a0bfcce0b].json @@ -0,0 +1,524 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 66600000.0, + "result_estimate_time": 0.44 + }, + "results": [ + { + "cite": "http://www.prouvecf.com", + "cmpt_rank": 0, + "details": { + "items": [ + { + "text": "HOME — PROUVE ...", + "url": "http://www.prouvecf.com/home" + }, + { + "text": "More results from prouvecf.com »", + "url": "/search?q=prouve+site:prouvecf.com&sca_esv=52b056e1a97f92b3&sa=X&ved=2ahUKEwiR9NGyjvCEAxWxD1kFHWdmCacQrAN6BAgTEAE" + } + ], + "type": "hyperlinks" + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "submenu", + "text": "Our products demonstrate good measures of well-being and positivism in light of modern innovative and sustainable designs. girosoft.jpg.", + "title": "PROUVE CONTRACT FURNITURE", + "type": "general", + "url": "http://www.prouvecf.com/" + }, + { + "cite": "https://en.wikipedia.org › wiki › Jean_Prouvé", + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Jean Prouvé was a French metal worker, self-taught architect and designer. Le Corbusier designated Prouvé a constructeur, blending architecture and ...", + "title": "Jean Prouvé", + "type": "general", + "url": "https://en.wikipedia.org/wiki/Jean_Prouv%C3%A9" + }, + { + "cite": "Design Within Reach", + "cmpt_rank": 2, + "details": { + "price": "$3,944.00", + "source": "Design Within Reach", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Prouvé Fauteuil de Salon. SALE.", + "type": "products", + "url": null + }, + { + "cite": "lichennyc.com", + "cmpt_rank": 2, + "details": { + "price": "$4,499.00", + "source": "lichennyc.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Jean Prouvé Tout Bois Chair.", + "type": "products", + "url": null + }, + { + "cite": "njmodern.com", + "cmpt_rank": 2, + "details": { + "price": "$819.00", + "source": "njmodern.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "prouve bridge director armchair - black.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 2, + "details": { + "price": "$137.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Jean Prouvé.", + "type": "products", + "url": null + }, + { + "cite": "finnishdesignshop.com", + "cmpt_rank": 2, + "details": { + "n_reviews": "4", + "price": "$902.70", + "rating": "5.0", + "source": "finnishdesignshop.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Petite Potence Wall Lamp Vitra. Low price.", + "type": "products", + "url": null + }, + { + "cite": "Design Within Reach", + "cmpt_rank": 2, + "details": { + "price": "$3,123.75", + "source": "Design Within Reach", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Prouvé Fauteuil Direction Design Within Reach. SALE.", + "type": "products", + "url": null + }, + { + "cite": "advancedinteriordesigns.com", + "cmpt_rank": 2, + "details": { + "price": "$140.00", + "source": "advancedinteriordesigns.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Prouve Style Standard Chair.", + "type": "products", + "url": null + }, + { + "cite": "njmodern.com", + "cmpt_rank": 2, + "details": { + "price": "$384.00", + "source": "njmodern.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "prouve standard chair.", + "type": "products", + "url": null + }, + { + "cite": "https://www.prouve.com", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Zestawy 90 dni; Perfumy. Nowości · Bestsellery · Edycje Limitowane · Perfumy Damskie · Perfumy Klasyczne · Precious Collection · Unique Collection ...", + "title": "Prouvé", + "type": "general", + "url": "https://www.prouve.com/" + }, + { + "cite": null, + "cmpt_rank": 4, + "details": { + "items": [ + "What was Jean Prouve's philosophy?", + "What style is Jean Prouve?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "What was Jean Prouve's philosophy?<|>What style is Jean Prouve?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.patrickseguin.com › furniture-jean-prouve", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "1929 Founding member of the Union des artistes modernes (UAM). First furniture: adjustable reclining armchair, pedestal table with rubber top, folding reclining ...", + "title": "Jean Prouvé - Furniture", + "type": "general", + "url": "https://www.patrickseguin.com/en/designers/furniture-jean-prouve/available-pieces-furniture-jean-prouve/" + }, + { + "cite": "https://www.vitra.com › designer › details › jean-prouve", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": null, + "text": "Prouvé's work encompasses a wide range of objects, from a letter opener to door and window fittings, from lighting and furniture to façade elements and ...", + "title": "Jean Prouvé | Official Vitra® Online Shop US", + "type": "general", + "url": "https://www.vitra.com/en-us/product/designer/details/jean-prouve" + }, + { + "cite": "https://www.dwr.com › Designers", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": null, + "text": "After opening his own workshop in 1923, Prouvé began producing modern metal furniture of his own design as well as collaborating with some of the best-known ...", + "title": "Jean Prouvé - Designers", + "type": "general", + "url": "https://www.dwr.com/designer-jean-prouv?lang=en_US" + }, + { + "cite": "https://www.jeanprouve.com › furniture", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": null, + "text": "1929. Articulated armchair ; 1929-30. Reclining armchair ; 1930. Grand Repos armchair ; 1930. Cité armchair ; 1937. Lawn chair with Jacques André.", + "title": "Furniture - Jean Prouvé", + "type": "general", + "url": "https://www.jeanprouve.com/en/furniture" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 0, + "sub_type": "shopping", + "text": "Shop deals", + "title": "Save with deals on apparel, electronics, and more", + "type": "promo", + "url": "/search?sca_esv=52b056e1a97f92b3&q=shop+deals&sa=X&ved=2ahUKEwiR9NGyjvCEAxWxD1kFHWdmCacQ_hF6BAhiEAA" + }, + { + "cite": "https://hivemodern.com › pages", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": null, + "text": "Jean Prouve's modern designs reveal knowledge of materials at hand & a commitment to collaboration between artist & craftsmen. Expert service and advice.", + "title": "Jean Prouvé", + "type": "general", + "url": "https://hivemodern.com/pages/category294/jean-prouve" + }, + { + "cite": "Amazon.com", + "cmpt_rank": 11, + "details": { + "price": "$173.67", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Jean Prouve - 5 Volume Box Set.", + "type": "products", + "url": null + }, + { + "cite": "prooffice.com", + "cmpt_rank": 11, + "details": { + "n_reviews": "1", + "price": "$326.72", + "rating": "3.0", + "source": "prooffice.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Prouve Antony Vitra Miniature Chair 1950.", + "type": "products", + "url": null + }, + { + "cite": "Design Within Reach", + "cmpt_rank": 11, + "details": { + "price": "$4,857.75", + "source": "Design Within Reach", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Prouvé Cité Lounge Chair. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Chairish", + "cmpt_rank": 11, + "details": { + "price": "$5,746.00", + "source": "Chairish", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Vintage Chairs by Jean Prouvé, 1941, Set of 2.", + "type": "products", + "url": null + }, + { + "cite": "finnishdesignshop.com", + "cmpt_rank": 11, + "details": { + "price": "$3,951.90", + "source": "finnishdesignshop.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "EM Table 240 x 90 cm, asphalt - Prouvé Blanc Colombe.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 11, + "details": { + "price": "$375.00", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Jean Prouve English and French Edition by Prouve Jean. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Pamono.com", + "cmpt_rank": 11, + "details": { + "price": "$5,711.00", + "source": "Pamono.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Vintage Large Coffee Table in the Style of Jean Prouvé, 1950s.", + "type": "products", + "url": null + }, + { + "cite": "Design Within Reach", + "cmpt_rank": 11, + "details": { + "price": "$1,164.50", + "source": "Design Within Reach", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Prouvé Tabouret Haut at Design Within Reach. SALE.", + "type": "products", + "url": null + }, + { + "cite": "https://www.finnishdesignshop.com › Home › Designers", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 0, + "sub_type": null, + "text": "Jean Prouvé - Designers - The most comprehensive selection of Finnish and Scandinavian design online. All in-stock items ship within 24 hours!", + "title": "Jean Prouvé | Design", + "type": "general", + "url": "https://www.finnishdesignshop.com/en-us/designer/jean-prouve" + }, + { + "cite": null, + "cmpt_rank": 13, + "details": { + "heading": "Related searches", + "items": [ + "prouvé meaning", + "prouvé perfume", + "prouvé chair", + "prouvé table", + "prouvé vitra", + "prouvé pronunciation", + "prouvé house", + "prouvé desk" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 27, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "prouvé meaning<|>prouvé perfume<|>prouvé chair<|>prouvé table<|>prouvé vitra<|>prouvé pronunciation<|>prouvé house<|>prouvé desk", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 14, + "details": { + "subtitle": "French architect", + "type": "panel" + }, + "error": null, + "section": "rhs", + "serp_rank": 28, + "sub_rank": 0, + "sub_type": "panel_rhs", + "text": "Jean Prouvé was a French metal worker, self-taught architect and designer. Le Corbusier designated Prouvé a constructeur, blending architecture and engineering. Prouvé's main achievement was transferring manufacturing technology from industry to architecture, without losing aesthetic qualities.", + "title": "Jean Prouvé", + "type": "knowledge", + "url": "https://en.wikipedia.org/wiki/Jean_Prouv%C3%A9" + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[363db5067378].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[363db5067378].json new file mode 100644 index 00000000..486ac07e --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[363db5067378].json @@ -0,0 +1,238 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": false, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 1150000.0, + "result_estimate_time": 0.3 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "unavailable", + "text": null, + "title": null, + "type": "ai_overview", + "url": null + }, + { + "cite": "https://www.honeywell.com › company › leadership", + "cmpt_rank": 1, + "details": { + "heading": "Leadership", + "img_url": null, + "text": "Leadership | Honeywell Honeywell https://www.honeywell.com › company › leadership|Honeywell| › company › leadership|Honeywell| › company › leadership|About featured snippets|•|Feedback|Feedback", + "type": "panel", + "urls": [ + { + "text": "More items...", + "url": "https://www.honeywell.com/us/en/company/leadership" + }, + { + "text": "", + "url": "/url?url=https://support.google.com/websearch?p%3Dfeatured_snippets%26hl%3Den-US&rct=j&q=&esrc=s&opi=89978449&usg=AOvVaw0i6agUwizJt0H2lHOHxNHI&hl=en&sa=X&ved=2ahUKEwilw7TT-6-LAxWgtlYBHR8fL18QrpwBegQIFRAB" + }, + { + "text": "About featured snippets", + "url": "/url?url=https://support.google.com/websearch?p%3Dfeatured_snippets%26hl%3Den-US&rct=j&q=&esrc=s&opi=89978449&usg=AOvVaw0i6agUwizJt0H2lHOHxNHI&hl=en&sa=X&ved=2ahUKEwilw7TT-6-LAxWgtlYBHR8fL18QrpwBegQIFRAC" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "featured_snippet", + "text": "Leadership", + "title": "Leadership | Honeywell", + "type": "knowledge", + "url": "https://www.honeywell.com/us/en/company/leadership" + }, + { + "cite": "https://www.comparably.com › ... › Leadership", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "The Honeywell Executive Team is rated a \"D-\" and led by CEO Darius Adamczyk. Honeywell employees rate their Executive Team in the Bottom 10% of similar size ...", + "title": "Honeywell Executive Team", + "type": "general", + "url": "https://www.comparably.com/companies/honeywell/executive-team" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": { + "items": [ + "How much do Honeywell executives get paid?", + "Who is the management team of Honeywell?", + "Who are C level management?", + "What are Honeywell corporate values?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "How much do Honeywell executives get paid?<|>Who is the management team of Honeywell?<|>Who are C level management?<|>What are Honeywell corporate values?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.globaldata.com › All Companies", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "GlobalData provides latest information on Honeywell International Inc employees , including CEO details, executives and full board of directors.", + "title": "Honeywell International Inc: Executives", + "type": "general", + "url": "https://www.globaldata.com/company-profile/honeywell-international-inc/executives/" + }, + { + "cite": "https://executivebiz.com › Articles", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "Sep 29, 2022 — List of Honeywell International Leaders and Executives · Darius Adamczyk, Chief Executive Officer and Chairman · Kevin Dehoff, President and Chief ...", + "title": "Who Are the Honeywell International Leaders and ...", + "type": "general", + "url": "https://executivebiz.com/articles/who-are-the-honeywell-international-leaders-and-executives/" + }, + { + "cite": "https://potomacofficersclub.com › articles › honeywell-i...", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "Oct 12, 2022 — Darius Adamczyk is Honeywell's Chief Executive Officer and Chairman . He is responsible for bringing technology and solutions to the markets to ...", + "title": "Honeywell International Leaders, Founders, And Executives", + "type": "general", + "url": "https://potomacofficersclub.com/articles/honeywell-international-leaders-founders-and-executives-who-are-they/" + }, + { + "cite": "https://craft.co › Honeywell International", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "Honeywell International CEO and Key Executive Team · Vimal Kapur · Billal M. Hammoud · Lucian Boldea · Kevin Dehoff · Anant Maheshwari · Ken West · Jim Currier · Sheila ...", + "title": "Honeywell International CEO and Key Executive Team", + "type": "general", + "url": "https://craft.co/honeywell-international/executives" + }, + { + "cite": "https://www.govconwire.com › articles › honeywell-inter...", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "Oct 12, 2022 — Honeywell International Executive Team · Darius Adamczyk, Chief Executive Officer and Chairman · Kevin Dehoff, President and Chief Executive ...", + "title": "Honeywell International Leaders, Founders, Executives", + "type": "general", + "url": "https://www.govconwire.com/articles/honeywell-international-leaders-founders-executives-who-are-they/" + }, + { + "cite": "https://www.honeywell.com › company › leadership › vi...", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Vimal Kapur is Chairman and CEO of Honeywell . After more than three decades rising through the leadership ranks, he became CEO in June 2023 and Chairman in ...", + "title": "Vimal Kapur", + "type": "general", + "url": "https://www.honeywell.com/us/en/company/leadership/vimal-kapur" + }, + { + "cite": "https://simplywall.st › stocks › nasdaq-hon › management", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "total yearly compensation is $14.40M , comprised of 8.5% salary and 91.5% bonuses, including company stock and options. directly owns 0.004% of the company's ...", + "title": "Honeywell International Inc. (HON) Leadership & ...", + "type": "general", + "url": "https://simplywall.st/stocks/us/capital-goods/nasdaq-hon/honeywell-international/management" + }, + { + "cite": "https://www.honeywell.com › press › 2023/03 › honeyw...", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "Mar 14, 2023 — Under Adamczyk's leadership, from 2016 through 2022, Honeywell's market capitalization grew from $88 billion to $145 billion, a 9% CAGR. Over ...", + "title": "Adamczyk To Serve As Executive Chairman", + "type": "general", + "url": "https://www.honeywell.com/us/en/press/2023/03/honeywell-names-34-year-honeywell-veteran-vimal-kapur-to-succeed-darius-adamczyk-as-ceo-on-june-1-2023-kapur-elected-to-company-s-board-of-directors-adamczyk-to-serve-as-executive-chairman" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": { + "heading": "People also search for", + "items": [ + "Honeywell c level management figures salary", + "Honeywell c level management figures list", + "Honeywell c level management figures 2022", + "Honeywell c level management figures 2021", + "Honeywell CEO history", + "Honeywell c level management figures 2020", + "Honeywell CEO salary", + "Honeywell executive compensation" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": "people_also_search_for", + "text": "Honeywell c level management figures salary<|>Honeywell c level management figures list<|>Honeywell c level management figures 2022<|>Honeywell c level management figures 2021<|>Honeywell CEO history<|>Honeywell c level management figures 2020<|>Honeywell CEO salary<|>Honeywell executive compensation", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[3c09a0f0c92f].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[3c09a0f0c92f].json deleted file mode 100644 index 6ba005dd..00000000 --- a/tests/__snapshots__/test_parse_serp/test_parse_serp[3c09a0f0c92f].json +++ /dev/null @@ -1,390 +0,0 @@ -{ - "features": { - "captcha": false, - "infinity_scroll": false, - "language": "en", - "main_layout": "standard", - "notice_no_results": false, - "notice_server_error": false, - "notice_shortened_query": false, - "overlay_precise_location": false, - "result_estimate_count": 55.0, - "result_estimate_time": 0.62 - }, - "results": [ - { - "cite": null, - "cmpt_rank": 0, - "details": { - "citations": [ - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "10", - "11", - "30", - "36", - "18", - "40" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "35", - "32", - "4" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "54", - "58", - "52", - "55", - "57" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "29", - "7", - "17" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "14" - ] - } - ], - "sources": [ - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.nejm.org&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "The New England Journal of Medicine", - "snippet": "Abstract * Background. Changes in the vaccine advisory process in the United States have disrupted immunization guidance, which re...", - "source_id": "10", - "title": "Updated Evidence for Covid-19, RSV, and Influenza Vaccines ...", - "url": "https://www.nejm.org/doi/full/10.1056/NEJMsa2514268#:~:text=The%20RSVpreF%20vaccine%20was%20associated,and%20the%20Alumbra%20Innovations%20Foundation.)" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://publichealth.jhu.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Johns Hopkins Bloomberg School of Public Health", - "snippet": "Is this year's flu vaccine effective against the flu viruses circulating this year? Yes, this year's flu vaccine provides effectiv...", - "source_id": "17", - "title": "How Bad Will This Winter Be for Flu, COVID, RSV, and Measles?", - "url": "https://publichealth.jhu.edu/2025/virus-transmission-trends-winter-2025-26#:~:text=Is%20this%20year's%20flu%20vaccine,%2Drelated%20hospitalizations%20this%20season.%E2%80%9D" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://oversight.house.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "House.gov", - "snippet": "WASHINGTON — The Select Subcommittee on the Coronavirus Pandemic held a hearing titled “Assessing America's Vaccine Safety Systems...", - "source_id": "52", - "title": "Hearing Wrap Up: Americans Deserve Improved Vaccine ...", - "url": "https://oversight.house.gov/release/hearing-wrap-up-americans-deserve-improved-vaccine-injury-and-compensation-systems/#:~:text=WASHINGTON%20%E2%80%94%20The%20Select%20Subcommittee%20on,a%20future%20public%20health%20crisis." - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.congress.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Congress.gov", - "snippet": "Table_title: Senate Committee Meeting Table_content: header: | Committee: | Senate Homeland Security and Governmental Affairs | ro...", - "source_id": "54", - "title": "Hearings to examine voices of the vaccine injured.", - "url": "https://www.congress.gov/event/119th-congress/senate-event/337242#:~:text=Hearings%20to%20examine%20voices%20of,Congress.gov%20%7C%20Library%20of%20Congress" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.sciencedirect.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "ScienceDirect.com", - "snippet": "Results. Pfizer and Moderna mRNA COVID-19 vaccines were associated with an excess risk of serious adverse events of special intere...", - "source_id": "40", - "title": "Serious adverse events of special interest following mRNA ...", - "url": "https://www.sciencedirect.com/science/article/pii/S0264410X22010283#:~:text=Pfizer%20and%20Moderna%20mRNA%20COVID%2D19%20vaccines%20were%20associated%20with,of%20serious%20COVID%2D19%20outcomes." - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.congress.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Congress.gov", - "snippet": "Dr. David Gortler is a pharmacist and pharmacologist who has experience as an investigational medicine scientist at Pfizer, as a p...", - "source_id": "55", - "title": "assessing america's vaccine safety systems part ii hearing", - "url": "https://www.congress.gov/118/meeting/house/117004/documents/HHRG-118-VC00-Transcript-20240321.pdf" - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.hrsa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Health Resources and Services Administration | HRSA (.gov)", - "snippet": "How does the VICP work? The National Vaccine Injury Compensation Program is a no-fault alternative to the traditional legal system...", - "source_id": "57", - "title": "National Vaccine Injury Compensation Program - HRSA", - "url": "https://www.hrsa.gov/vaccine-compensation#:~:text=How%20does%20the%20VICP%20work,and%20individuals%20who%20are%20deceased." - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.uchealth.org&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "UCHealth", - "snippet": "To answer some of your top questions about the 2025-2026 COVID-19 and flu vaccines, we consulted with infectious disease expert, D...", - "source_id": "14", - "title": "What you need to know about the 2025-26 COVID-19 vaccine ...", - "url": "https://www.uchealth.org/today/everything-you-need-to-know-about-the-2025-26-covid-19-vaccine-and-flu-shots/#:~:text=To%20answer%20some%20of%20your,the%20late%20fall%20and%20winter." - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.congress.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Congress.gov", - "snippet": "Hearings to examine science and Federal health agencies, focusing on Myocarditis and other events associated with the COVID-19 vac...", - "source_id": "58", - "title": "Senate Committee Meeting - Congress.gov", - "url": "https://www.congress.gov/event/119th-congress/senate-event/336985#:~:text=Senate%20Event%20336985-,Hearings%20to%20examine%20science%20and%20Federal%20health%20agencies%2C%20focusing%20on,119th%20Congress%20(2025%2D2026)" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://biontechse.gcs-web.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "BioNTech", - "snippet": "In clinical studies, adverse reactions in participants 16 years of age and older included pain at the injection site (84.1%), fati...", - "source_id": "36", - "title": "Pfizer and BioNTech Publish Preclinical Data from ...", - "url": "https://biontechse.gcs-web.com/news-releases/news-release-details/pfizer-and-biontech-publish-preclinical-data-investigational/#:~:text=In%20clinical%20studies%2C%20adverse%20reactions,to%20complete%20the%20vaccination%20series." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.cdc.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Centers for Disease Control and Prevention | CDC (.gov)", - "snippet": "Some of the observed symptoms and conditions may include insomnia, chronic pain and fatigue, dysautonomia (e.g., POTS), immune dys...", - "source_id": "18", - "title": "COVID-19 Vaccine Discussion Framing, 2025-2026 - CDC", - "url": "https://www.cdc.gov/acip/downloads/slides-2025-09-18-19/10-levi-COVID-508.pdf" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.cidrap.umn.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "CIDRAP", - "snippet": "Current flu vaccine provides moderate protection against severe disease, interim analyses suggest * Two new analyses, one from Fra...", - "source_id": "11", - "title": "Current flu vaccine provides moderate protection against ...", - "url": "https://www.cidrap.umn.edu/influenza-vaccines/current-flu-vaccine-provides-moderate-protection-against-severe-disease-interim#:~:text=Two%20new%20analyses%2C%20one%20from,in%20the%202024%E2%80%9325%20season." - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://cardiovascularbusiness.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Cardiovascular Business", - "snippet": "Cardiologist Peter McCullough no stranger to the COVID-related controversy. McCullough has gained significant popularity since the...", - "source_id": "35", - "title": "Cardiologist’s false claims used to promote fake COVID-19 vaccine ...", - "url": "https://cardiovascularbusiness.com/topics/clinical/covid-19/cardiologists-false-claims-used-promote-fake-covid-19-vaccine-recall-fact#:~:text=Cardiologist%20Peter%20McCullough%20no%20stranger,his%20claims%20as%20%22misleading.%22" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.cidrap.umn.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "CIDRAP", - "snippet": "Major themes were disputing COVID-19 vaccine safety and effectiveness, promoting non–evidence-based medical treatments or those la...", - "source_id": "29", - "title": "Report spotlights 52 US doctors who posted potentially ...", - "url": "https://www.cidrap.umn.edu/covid-19/report-spotlights-52-us-doctors-who-posted-potentially-harmful-covid-misinformation-online#:~:text=Major%20themes%20were%20disputing%20COVID,inflammation%20of%20the%20heart%20muscle)." - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.healthychildren.org&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "HealthyChildren.org", - "snippet": "Research continues to confirm that vaccines are safe and effective—and they protect children and teens from serious diseases.", - "source_id": "7", - "title": "Vaccine Safety: Examine the Evidence - HealthyChildren.org", - "url": "https://www.healthychildren.org/English/safety-prevention/immunizations/Pages/vaccine-studies-examine-the-evidence.aspx#:~:text=Research%20continues%20to%20confirm%20that,and%20teens%20from%20serious%20diseases." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.youtube.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "YouTube", - "snippet": "and what people are denying. they're not denying the potential effectiveness of vaccines i I think most of us think yeah they prob...", - "source_id": "30", - "title": "'Vaccine Didn't End Covid...': Dr Toby Rogers' EXPLOSIVE ...", - "url": "https://www.youtube.com/watch?v=aafUmBFS42Q" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.nbcnews.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "NBC News", - "snippet": "Unless state medical boards begin to hold these physicians accountable, the trust of the entire medical profession is in jeopardy.", - "source_id": "32", - "title": "Covid vaccine and treatment misinformation is medical ...", - "url": "https://www.nbcnews.com/think/opinion/covid-vaccine-treatment-misinformation-medical-malpractice-it-should-be-punished-ncna1287180#:~:text=But%20a%20vocal%20minority%20of,to%20hold%20these%20doctors%20accountable." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.youtube.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "YouTube", - "snippet": "here our hearing was actively suppressed scrubbed off YouTube. so just just so just so just to clarify. not only were your viewpoi...", - "source_id": "4", - "title": "LIVE: 'COVID Vaccines caused 74% deaths': Dr. McCullough's ...", - "url": "https://www.youtube.com/watch?v=oZ1hgVNRMmE" - } - ], - "type": "ai_overview" - }, - "error": null, - "section": "main", - "serp_rank": 0, - "sub_rank": 0, - "sub_type": "flat", - "text": "Based on recent reports, congressional testimony, and medical updates, the landscape surrounding COVID-19 and influenza vaccines in late 2025 and early 2026 continues to be polarized, featuring both high vaccination effectiveness data and ongoing concerns regarding rare side effects and, in some circles, allegations of cover-ups. Medical Evidence, Safety, and Ingredients (2025-2026) Effectiveness: Updated 2025-2026 COVID-19 mRNA vaccines for the KP.2 subvariant showed 68% effectiveness against hospitalization, while influenza vaccines provided moderate protection, with effectiveness higher in children (57.2%) than in older adults (25.3%–27.7%). Safety Profile: Major, peer-reviewed evidence continues to support the safety of vaccines, though rare risks like myocarditis (1.3 to 3.1 per 100,000 doses in male adolescents) are documented. Ingredients/Mechanism: mRNA vaccines (Pfizer/Moderna) utilize modified mRNA to instruct cells to produce a spike protein, triggering an immune response. Concerns/Injuries: Some studies have suggested that spike proteins or lipid nanoparticles might persist in the body, potentially contributing to rare \"post-vaccine syndrome\" (PVS). Data indicates serious adverse events of special interest, such as clotting issues, were higher in mRNA groups than placebo in initial trials. Pregnancy/Safety: Data on COVID-19 vaccine safety in pregnancy has been described as insufficient in some clinical trials, though public health guidance generally recommends them. Doctor Interviews, Truth, and \"Coverup\" Allegations Divergent Views: A vocal minority of doctors, such as Dr. Peter McCullough, have claimed that federal agencies, such as the CDC, purposefully concealed vaccine risks or withheld early treatments, a stance frequently labeled as misleading by major medical institutions. \"Disinformation\" Doctors: Conversely, reports have identified a \"cottage industry\" of doctors spreading unproven treatments (e.g., ivermectin) and misinformation about vaccine ingredients, often using their professional status to build social media followings. Suppression Claims: Some medical professionals have testified in Senate hearings that their findings regarding vaccine injuries were actively suppressed or dismissed as conspiracy theories. Congress, Trials, and Investigations Congressional Hearings: The 119th Congress (2025-2026) has continued to hold hearings investigating the \"voices of the vaccine injured,\" focusing on myocarditis, and reviewing the adequacy of CDC/FDA safety surveillance systems. Safety Systems: Testimony has suggested deficiencies in vaccine safety reporting (VAERS), with some witnesses arguing that systems were not designed to adequately track long-term adverse events. Liability: The National Vaccine Injury Compensation Program (VICP) is the established venue for injury claims, designed as a no-fault alternative to civil litigation. Influencer vs. Expert Narratives Vaccine Skepticism: Anti-vax influencers and alternative media figures frequently highlight individual anecdotes of injury to argue that the vaccines are unsafe, often focusing on the speed of development. Pro-Vaccine Stance: Health agencies (CDC) and major medical institutions maintain that vaccines are the most effective tool for preventing death and severe disease, stating that the benefits far outweigh the risks. Global Regime and Future Plan (USA) Ongoing Vaccination: The 2025-2026 strategy in the USA, in line with international bodies, continues to encourage annual, updated boosters for influenza and COVID-19 to manage seasonal surges. Policy Shifts: Changes in the vaccine advisory process in the U.S. have led to calls for more independent evidence reviews regarding future vaccine requirements. Disclaimer: Medical information changes rapidly. This summary reflects data reported up to early 2026. Always consult a licensed healthcare provider for personal medical advice.", - "title": null, - "type": "ai_overview", - "url": null - }, - { - "cite": "1.3K+ views · 2 weeks ago", - "cmpt_rank": 1, - "details": null, - "error": null, - "section": "main", - "serp_rank": 1, - "sub_rank": 0, - "sub_type": null, - "text": "Dr . Glen Jeffery: Using Red Light to Improve Your Health & the Harmful Effects of LEDs. Andrew Huberman•255K views · 19:01. Go to channel Legal ... Read more", - "title": "Everything You're Afraid to Ask About Vaccines", - "type": "general", - "url": "https://www.youtube.com/watch?v=fVxNCWm3_cs" - }, - { - "cite": null, - "cmpt_rank": 2, - "details": { - "items": [ - "Why are people refusing the COVID vaccine?", - "Stanford Medicine study shows why mRNA COVID-19 vaccine can ... Stanford Medicine - Stanford University https://med.stanford.edu › myocarditis-vaccine-covid Stanford Medicine - Stanford University https://med.stanford.edu › myocarditis-vaccine-covid", - "What percentage of people have a serious side effect from the COVID vaccine?", - "Adverse Effects of Pfizer (BioNTech), Oxford-AstraZeneca (ChAdOx1 ... PMC - NIH https://pmc.ncbi.nlm.nih.gov › articles › PMC9967558 PMC - NIH https://pmc.ncbi.nlm.nih.gov › articles › PMC9967558" - ], - "type": "text" - }, - "error": null, - "section": "main", - "serp_rank": 2, - "sub_rank": 0, - "sub_type": null, - "text": "Why are people refusing the COVID vaccine?<|>Stanford Medicine study shows why mRNA COVID-19 vaccine can ... Stanford Medicine - Stanford University https://med.stanford.edu › myocarditis-vaccine-covid Stanford Medicine - Stanford University https://med.stanford.edu › myocarditis-vaccine-covid<|>What percentage of people have a serious side effect from the COVID vaccine?<|>Adverse Effects of Pfizer (BioNTech), Oxford-AstraZeneca (ChAdOx1 ... PMC - NIH https://pmc.ncbi.nlm.nih.gov › articles › PMC9967558 PMC - NIH https://pmc.ncbi.nlm.nih.gov › articles › PMC9967558", - "title": null, - "type": "people_also_ask", - "url": null - }, - { - "cite": "https://www.ideastream.org › lying-through-truth-misle...", - "cmpt_rank": 3, - "details": null, - "error": null, - "section": "main", - "serp_rank": 3, - "sub_rank": 0, - "sub_type": null, - "text": "Mar 25, 2021 — ... of an effort to take down patently false health claims, bad actors are turning to cherry-picked truths to drive misleading narratives. Read more", - "title": "'Lying Through Truth': Misleading Facts Fuel Vaccine ...", - "type": "general", - "url": "https://www.ideastream.org/2021-03-25/lying-through-truth-misleading-facts-fuel-vaccine-misinformation" - }, - { - "cite": "2.4K+ views · 8 months ago", - "cmpt_rank": 4, - "details": null, - "error": null, - "section": "main", - "serp_rank": 4, - "sub_rank": 0, - "sub_type": null, - "text": "The title of today's hearing is the corruption of science and federal health agencies how health officials downplayed and hid myocarditis.", - "title": "US Senate LIVE | Covid Vaccine 'TRUTH BOMB' Shocks USA", - "type": "general", - "url": "https://www.youtube.com/watch?v=1P1Ui2eEjPw" - }, - { - "cite": "9.2K+ views · 1 year ago", - "cmpt_rank": 5, - "details": null, - "error": null, - "section": "main", - "serp_rank": 5, - "sub_rank": 0, - "sub_type": null, - "text": "The intent to vaccinating the United States was reduced by nearly three percentage points due to unflagged vaccine skeptical content.", - "title": "Covid Vaccine Misinformation or Disinformation? Which ...", - "type": "general", - "url": "https://www.youtube.com/watch?v=jLieri_aweE" - }, - { - "cite": "2.1M+ views · 5 years ago", - "cmpt_rank": 6, - "details": null, - "error": null, - "section": "main", - "serp_rank": 6, - "sub_rank": 0, - "sub_type": null, - "text": "(Dec. 8) Watch live as the U.S. Senate examines Covid -19 solutions, focusing on early outpatient treatment in Washington, D.C., on Tuesday, ...", - "title": "Anti-Vaccine Doctor Testifies at Senate Homeland Security ...", - "type": "general", - "url": "https://www.youtube.com/watch?v=k8RyV3VEDKI" - }, - { - "cite": "98.7K+ views · 1 year ago", - "cmpt_rank": 7, - "details": null, - "error": null, - "section": "main", - "serp_rank": 7, - "sub_rank": 0, - "sub_type": null, - "text": "I sat down with Dr . Jay Bhattacharya, professor of medicine at Stanford University and a leading voice in public health policy, to explore ...", - "title": "Trump's NIH Pick Dr. Jay Bhattacharya on COVID Lies ...", - "type": "general", - "url": "https://www.youtube.com/watch?v=eO3mOUyk5FE" - }, - { - "cite": "https://www.factcheck.org › 2024/01 › faulty-science-u...", - "cmpt_rank": 8, - "details": null, - "error": null, - "section": "main", - "serp_rank": 8, - "sub_rank": 0, - "sub_type": null, - "text": "Jan 5, 2024 — Getting the latest vaccine provides additional protection against poor outcomes from COVID -19. Serious side effects are rare, and the  ... Read more", - "title": "Faulty Science Underpins Florida Surgeon General's Call ...", - "type": "general", - "url": "https://www.factcheck.org/2024/01/faulty-science-underpins-florida-surgeon-generals-call-to-halt-mrna-covid-19-vaccination/" - }, - { - "cite": "1.4K+ views · 2 months ago", - "cmpt_rank": 9, - "details": null, - "error": null, - "section": "main", - "serp_rank": 9, - "sub_rank": 0, - "sub_type": null, - "text": "LIVE: ' COVID Vaccines caused 74% deaths ': Dr . McCullough's Shocking Senate Statement Goes Viral #DrMcCullough #COVIDVaccineRisks ...", - "title": "Dr. McCullough's Shocking Senate Statement Goes Viral", - "type": "general", - "url": "https://www.youtube.com/watch?v=oZ1hgVNRMmE" - }, - { - "cite": "1.6K+ views · 1 month ago", - "cmpt_rank": 10, - "details": null, - "error": null, - "section": "main", - "serp_rank": 10, - "sub_rank": 0, - "sub_type": null, - "text": "We have five witnesses who will tell how drug and vaccine injuries have impacted their lives.", - "title": "'They knew the risks, yet…': Shocking COVID vaccine side ...", - "type": "general", - "url": "https://www.youtube.com/watch?v=z6U07WOas8E" - }, - { - "cite": "https://dokumen.pub › religious-dimensions-of-conspir...", - "cmpt_rank": 11, - "details": null, - "error": null, - "section": "main", - "serp_rank": 11, - "sub_rank": 0, - "sub_type": null, - "text": "... good ' and ' bad ' Freemasonry according to the criterion of Masonic regularity. ... the influence of evil forces contribute to put the puzzle together. Such ... Read more", - "title": "Religious Dimensions of Conspiracy Theories", - "type": "general", - "url": "https://dokumen.pub/religious-dimensions-of-conspiracy-theories-comparing-and-connecting-old-and-new-trends-2022023914-2022023915-9780367637705-9780367638436-9781003120940.html" - } - ] -} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[3f99953e830e].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[3f99953e830e].json new file mode 100644 index 00000000..7a0e3b3b --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[3f99953e830e].json @@ -0,0 +1,1011 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 8450000.0, + "result_estimate_time": 0.48 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "header", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "location_choose_area", + "text": null, + "title": "Results for United States", + "type": "notice", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "n_reviews": "9", + "price": "$154.99", + "source": "Cabela's", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "product", + "text": null, + "title": "Kelly Kettle - Ultimate Stainless Steel Scout Cook Kit for $154.99 from Cabela's", + "type": "shopping_ads", + "url": "https://www.cabelas.com/shop/en/kelly-kettle-ultimate-stainless-steel-scout-cook-kit" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$89.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 1, + "sub_type": "product", + "text": null, + "title": "Kelly Kettle - Camp Stove Stainless Steel - Boils Water Within Minutes, Uses Natural Fuel, and Enables You to Rehydrate Food or Cook a Meal (Medium for $89.99 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/Camp-Stove-Stainless-Minutes-Rehydrate/dp/B00AQ7YA26?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A1DFXHFO2LOYV7" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$169.99", + "source": "Kelly Kettle USA", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 2, + "sub_type": "product", + "text": null, + "title": "Kelly Kettle - VALUE DEAL - Ultimate 'Base Camp' Kelly Kettle Kit (S.Steel) Includes 54oz Camping Kettle, Cook Set, Cups, Plates, Hobo Stove & Pot Support for $169.99 from Kelly Kettle USA", + "type": "shopping_ads", + "url": "https://www.kellykettleusa.com/ultimate-stainless-base-camp-kit" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "n_reviews": "200", + "price": "$164.99", + "source": "Cabela's", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 3, + "sub_type": "product", + "text": null, + "title": "Kelly Kettle - Ultimate Stainless Steel Base Camp Cook Kit for $164.99 from Cabela's", + "type": "shopping_ads", + "url": "https://www.cabelas.com/shop/en/kelly-kettle-ultimate-stainless-steel-base-camp-cook-kit" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$179.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 4, + "sub_type": "product", + "text": null, + "title": "Kelly Kettle - Ultimate Base Camp Kit – 54 oz Large Stainless Steel Camp Kettle, Lightweight Camping Kettle with Whistle, Kelly Kettle Stove for for $179.99 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/Stainless-Ultimate-Emergencies-Hurricanes-Tornados/dp/B00OABV4I2?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A1DFXHFO2LOYV7" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$169.99", + "source": "Sagan Life", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 5, + "sub_type": "product", + "text": null, + "title": "Kelly Kettle - Ultimate Base Camp Kit – Stainless Steel Camp Kettle for $169.99 from Sagan Life", + "type": "shopping_ads", + "url": "https://saganlife.com/product/ultimate-stainless-base-camp-kit/" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$124.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 6, + "sub_type": "product", + "text": null, + "title": "Kelly Kettle - Base Camp Stainless Steel Basic Kit 54 oz.(1.6 LTR) Large Stainless Steel Camp Kettle, Lightweight Camping Kettle with Whistle, Camp for $124.99 from Amazon.com", + "type": "shopping_ads", + "url": "https://www.amazon.com/Kelly-Kettle-Large-Stainless-Steel/dp/B004GLCKZC?source=ps-sl-shoppingads-lpcontext&ref_=fplfs&psc=1&smid=A1DFXHFO2LOYV7" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "price": "$164.99", + "source": "Kelly Kettle USA", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 7, + "sub_type": "product", + "text": null, + "title": "Kelly Kettle - VALUE DEAL - Ultimate 'Scout' Kelly Kettle Kit (S.Steel) - Includes 41oz Camping Kettle, Cook Set, Cups, Plates, Hobo Camp Stove & Pot Support for $164.99 from Kelly Kettle USA", + "type": "shopping_ads", + "url": "https://www.kellykettleusa.com/ultimate-stainless-scout-kit" + }, + { + "cite": "https://www.kellykettleusa.com", + "cmpt_rank": 2, + "details": { + "items": [ + { + "text": "Kits", + "url": "https://www.kellykettleusa.com/kelly-kettle-kits" + }, + { + "text": "Kettles", + "url": "https://www.kellykettleusa.com/kelly-kettles" + }, + { + "text": "Stainless Steel 'Scout' Kettle ...", + "url": "https://www.kellykettleusa.com/stainless-medium-scout-kelly-kettler-complete-kit" + }, + { + "text": "Stainless Steel", + "url": "https://www.kellykettleusa.com/kelly-kettles/stainless-steel" + }, + { + "text": "More results from kellykettleusa.com »", + "url": "/search?q=kelly+kettle+site:kellykettleusa.com&sca_esv=52b056e1a97f92b3&sa=X&ved=2ahUKEwjm5pn9hfCEAxV-FlkFHWieCNQQrAN6BAgOEAE" + } + ], + "type": "hyperlinks" + }, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": "submenu", + "text": "Kelly Kettle Camping Gear | Boil water & cook fast outdoors | Ideal for scouts, kayaking, hiking, trekking, hunting, fishing, survival, preppers, ...", + "title": "Kelly Kettle Camping Kettle & Stove | Boil Water & Cook ...", + "type": "general", + "url": "https://www.kellykettleusa.com/" + }, + { + "cite": "https://www.kellykettle.com", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Enjoy hot water for tea, coffee, rehydrating food, bathing, cooking, etc. Works everywhere, every time! Believed to be the fastest Camping Kettle in the world, ...", + "title": "Kelly Kettle® Camping Kettle & Stove", + "type": "general", + "url": "https://www.kellykettle.com/" + }, + { + "cite": "https://www.amazon.com › Camp-Stove-Stainless-Minute...", + "cmpt_rank": 4, + "details": { + "price": "ating: 4.7", + "stock": " 217 reviews", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "The Stainless Steel Kelly Kettle is a multi-use camping stove that gives you the ultimate outdoor experience. You have the ability to boil and purify water ...", + "title": "Kelly Kettle Camp Stove Stainless Steel - Boils ...", + "type": "general", + "url": "https://www.amazon.com/Camp-Stove-Stainless-Minutes-Rehydrate/dp/B00AQ7YA26" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": { + "items": [ + "Is the Kelly Kettle worth it?", + "What does a Kelly Kettle do?", + "What is another name for Kelly Kettle?", + "How long does it take to boil water in a Kelly Kettle?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "Is the Kelly Kettle worth it?<|>What does a Kelly Kettle do?<|>What is another name for Kelly Kettle?<|>How long does it take to boil water in a Kelly Kettle?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "Varuste.net", + "cmpt_rank": 6, + "details": { + "n_reviews": "200", + "price": "$94.40", + "rating": "4.8", + "source": "Varuste.net", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Ultimate Base Camp Kit. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Varuste.net", + "cmpt_rank": 6, + "details": { + "n_reviews": "108", + "price": "$52.80", + "rating": "4.9", + "source": "Varuste.net", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Stainless Steel Base Camp. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Varuste.net", + "cmpt_rank": 6, + "details": { + "n_reviews": "111", + "price": "$50.69", + "rating": "4.9", + "source": "Varuste.net", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Stainless Steel Scout. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Varuste.net", + "cmpt_rank": 6, + "details": { + "n_reviews": "114", + "price": "$90.95", + "rating": "4.9", + "source": "Varuste.net", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Ultimate Stainless Steel Scout Kit. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 6, + "details": { + "n_reviews": "70", + "price": "$76.99", + "rating": "4.9", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Stainless Steel Trekker Kettle.", + "type": "products", + "url": null + }, + { + "cite": "Cabela's", + "cmpt_rank": 6, + "details": { + "n_reviews": "18", + "price": "$117.99", + "rating": "4.8", + "source": "Cabela's", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Stainless Steel Trekker & Kit.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 6, + "details": { + "n_reviews": "1", + "price": "$179.99", + "rating": "5.0", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Ultimate Base Camp Kit – 54 oz Large Stainless Steel Camp Kettle.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 6, + "details": { + "price": "$174.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Ultimate Scout Kit 41 oz. Stainless Steel Camp Kettle.", + "type": "products", + "url": null + }, + { + "cite": "https://www.kellykettle.com › camping-kettles", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": null, + "text": "KELLY CAMPING KETTLES · Base Camp - Boils 1.6 litres | Packed Height 33 cm | Kettle Width 15.5cm | Widest point of fire-base 18.5cm | Suits: Groups, Family ...", + "title": "Survival Kit | Kelly Kettle Camping Kettle & Stove | ...", + "type": "general", + "url": "https://www.kellykettle.com/camping-kettles" + }, + { + "cite": "https://en.wikipedia.org › wiki › Kelly_Kettle", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 0, + "sub_type": null, + "text": "Kelly Kettle , Storm Kettle, Ghillie Kettle, Thermette, Survival Kettle and Volcano Kettle are trade names for portable devices for boiling water outdoors ...", + "title": "Kelly Kettle", + "type": "general", + "url": "https://en.wikipedia.org/wiki/Kelly_Kettle" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 0, + "sub_type": "shopping", + "text": "Shop deals", + "title": "Save with deals on apparel, electronics, and more", + "type": "promo", + "url": "/search?sca_esv=52b056e1a97f92b3&q=shop+deals&sa=X&ved=2ahUKEwjm5pn9hfCEAxV-FlkFHWieCNQQ_hF6BAhzEAA" + }, + { + "cite": "https://getlost.com › collections › kelly-kettle", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 0, + "sub_type": null, + "text": "The Kelly Kettle , also known as the Ghillie Kettle or Volcano Stove, is designed to boil water outdoors using twigs, leaves, and other small combustible ...", + "title": "Kelly Kettles, Kits, and Accessories | Get Lost", + "type": "general", + "url": "https://getlost.com/collections/kelly-kettle" + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 11, + "details": { + "n_reviews": "67", + "price": "$124.99", + "rating": "4.7", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Base Camp – Basic Kit – Stainless Steel Camp Kettle.", + "type": "products", + "url": null + }, + { + "cite": "OpticsPlanet.com", + "cmpt_rank": 11, + "details": { + "n_reviews": "12", + "price": "$71.24", + "rating": "4.3", + "source": "OpticsPlanet.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Aluminum Scout. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 11, + "details": { + "n_reviews": "2", + "price": "$99.99", + "rating": "5.0", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Trekker Basic Kit – Stainless Steel Camping Kettle.", + "type": "products", + "url": null + }, + { + "cite": "OpticsPlanet.com", + "cmpt_rank": 11, + "details": { + "n_reviews": "3", + "price": "$75.35", + "rating": "5.0", + "source": "OpticsPlanet.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Aluminum Base Camp - Large. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 11, + "details": { + "n_reviews": "2", + "price": "$110.99", + "rating": "5.0", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Trekker Basic Kit.", + "type": "products", + "url": null + }, + { + "cite": "nickelsboatworks.com", + "cmpt_rank": 11, + "details": { + "price": "$109.97", + "source": "nickelsboatworks.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Kit.", + "type": "products", + "url": null + }, + { + "cite": "OpticsPlanet.com", + "cmpt_rank": 11, + "details": { + "n_reviews": "14", + "price": "$62.34", + "rating": "4.8", + "source": "OpticsPlanet.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Aluminum Trekker - Small. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Walmart - Kelly Kettle USA", + "cmpt_rank": 11, + "details": { + "n_reviews": "3", + "price": "$114.95", + "rating": "5.0", + "source": "Walmart - Kelly Kettle USA", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Scout Basic Kit (Medium) - Stainless Steel.", + "type": "products", + "url": null + }, + { + "cite": "readymaderesources.com", + "cmpt_rank": 12, + "details": { + "price": "$179.99", + "source": "readymaderesources.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 33, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Stainless Steel-Base Camp Kelly Kettle - Complete Kit.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 12, + "details": { + "price": "$114.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 34, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Scout Basic Kit.", + "type": "products", + "url": null + }, + { + "cite": "Varuste.net", + "cmpt_rank": 12, + "details": { + "n_reviews": "33", + "price": "$13.32", + "rating": "4.9", + "source": "Varuste.net", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 35, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Large Cook Set. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 12, + "details": { + "price": "$78.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 36, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Trekker 20 oz. Anodized Aluminum Rocket Stove.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 12, + "details": { + "n_reviews": "1", + "price": "$109.99", + "rating": "5.0", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 37, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Large Aluminum Base Camp Kelly Kettle Basic Kit.", + "type": "products", + "url": null + }, + { + "cite": "readymaderesources.com", + "cmpt_rank": 12, + "details": { + "price": "$109.99", + "source": "readymaderesources.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 38, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Medium Aluminum Scout Camp Stove Basic Kit..", + "type": "products", + "url": null + }, + { + "cite": "readymaderesources.com", + "cmpt_rank": 12, + "details": { + "price": "$68.00", + "source": "readymaderesources.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 39, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Aluminum-Scout-Medium Kelly Kettle.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 12, + "details": { + "n_reviews": "7", + "price": "$26.99", + "rating": "4.8", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 40, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Small Cook Set.", + "type": "products", + "url": null + }, + { + "cite": "https://www.amazon.com › Kelly-Kettle-Large-Stainless-...", + "cmpt_rank": 13, + "details": { + "price": "ating: 4.7", + "stock": " 166 reviews", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 41, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "The Kelly kettle stainless steel base camp basic kit is a highly efficient outdoor camping cook stove which is suitable for car camping, family picnics, ...", + "title": "Kelly Kettle Base Camp Stainless Steel Basic Kit ...", + "type": "general", + "url": "https://www.amazon.com/Kelly-Kettle-Large-Stainless-Steel/dp/B004GLCKZC" + }, + { + "cite": "https://saganlife.com › SHOP", + "cmpt_rank": 14, + "details": { + "price": "ating: 5", + "stock": " 19 reviews", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 42, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "The Kelly Kettle Scout is the medium size stainless steel camping kettle for camping, fishing, hunting, kayaking and outdoor adventures.", + "title": "Kelly kettle scout stainless steel camp kettle", + "type": "general", + "url": "https://saganlife.com/product/stainless-scout-medium-kelly-kettle/" + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 15, + "details": { + "n_reviews": "62", + "price": "$17.99", + "rating": "4.8", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 43, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Hobo Stove.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 15, + "details": { + "price": "$89.99", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 44, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle 50003 Medium Aluminum Scout Camp Stove.", + "type": "products", + "url": null + }, + { + "cite": "prepsos.com", + "cmpt_rank": 15, + "details": { + "price": "$98.98", + "source": "prepsos.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 45, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle ‘Base Camp’ 54 fl.oz. Camp Kettle (Stainless Steel) – (SHIPS IN 1-2 WEEKS).", + "type": "products", + "url": null + }, + { + "cite": "wellprepared.com", + "cmpt_rank": 15, + "details": { + "price": "$84.98", + "source": "wellprepared.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 46, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Stainless Steel Scout Kettle.", + "type": "products", + "url": null + }, + { + "cite": "Expedition Exchange", + "cmpt_rank": 15, + "details": { + "price": "$68.00", + "source": "Expedition Exchange", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 47, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Aluminum Base Camp (50002).", + "type": "products", + "url": null + }, + { + "cite": "Food Assets", + "cmpt_rank": 15, + "details": { + "price": "$76.95", + "source": "Food Assets", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 48, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Scout Cookstove.", + "type": "products", + "url": null + }, + { + "cite": "wellprepared.com", + "cmpt_rank": 15, + "details": { + "price": "$89.98", + "source": "wellprepared.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 49, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Stainless Steel Base Camp Kettle.", + "type": "products", + "url": null + }, + { + "cite": "Food Assets", + "cmpt_rank": 15, + "details": { + "price": "$65.55", + "source": "Food Assets", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 50, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Kelly Kettle Trekker Cookstove.", + "type": "products", + "url": null + }, + { + "cite": null, + "cmpt_rank": 16, + "details": { + "heading": "Related searches", + "items": [ + "kelly kettle trekker", + "kelly kettle scout", + "kelly kettle alternative", + "kelly kettle amazon", + "Kelly kettle website", + "kelly kettle hobo stove" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 51, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "kelly kettle trekker<|>kelly kettle scout<|>kelly kettle alternative<|>kelly kettle amazon<|>Kelly kettle website<|>kelly kettle hobo stove", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 17, + "details": null, + "error": null, + "section": "rhs", + "serp_rank": 52, + "sub_rank": 0, + "sub_type": "panel_rhs", + "text": null, + "title": "See results about", + "type": "knowledge", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[45b6e019bfa2].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[45b6e019bfa2].json deleted file mode 100644 index 486d7ed6..00000000 --- a/tests/__snapshots__/test_parse_serp/test_parse_serp[45b6e019bfa2].json +++ /dev/null @@ -1,512 +0,0 @@ -{ - "features": { - "captcha": false, - "infinity_scroll": false, - "language": "en", - "main_layout": "standard", - "notice_no_results": false, - "notice_server_error": false, - "notice_shortened_query": false, - "overlay_precise_location": false, - "result_estimate_count": 2980000000.0, - "result_estimate_time": 0.24 - }, - "results": [ - { - "cite": null, - "cmpt_rank": 0, - "details": { - "citations": [ - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "1", - "3", - "4" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "7", - "6", - "2", - "9", - "5", - "8" - ] - } - ], - "sources": [ - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.facebook.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Facebook", - "snippet": "The sky appears blue because of a phenomenon called Rayleigh scattering. Sunlight is made up of all the colors of the rainbow, eac...", - "source_id": "3", - "title": "Why is the sky blue? It's an age old question that actually has a very simple answer The sun emits a white light that consists of all the colors of the rainbow At about 18 miles above the earth the light begins to interact with air molecules Those air molecules are the perfect size to scatter the blue light wavelengths, but all other colors continue down to the surface The blue light is scattered from molecule to molecule and reflected at you And now you know 🙃", - "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.reddit.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Reddit", - "snippet": "The sky is blue because blue light gets scattered the most during the day. Rayleigh scattering states that the amount of scatterin...", - "source_id": "8", - "title": "Why is the sky blue? Do I understand it correctly: : r/askscience", - "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.nesdis.noaa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Environmental Satellite, Data, and Information Service NESDIS (.gov)", - "snippet": "Gases and particles in Earth's atmosphere scatter sunlight in all directions. Blue light is scattered more than other colors becau...", - "source_id": "1", - "title": "Why Is the Sky Blue? | NESDIS", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue#:~:text=Gases%20and%20particles%20in%20Earth's%20atmosphere%20scatter,a%20blue%20sky%20most%20of%20the%20time." - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://kids.nationalgeographic.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Geographic Kids", - "snippet": "The sky is blue because of the scattering of light in the atmosphere. The sun's light is made up of all the colors of the rainbow,", - "source_id": "6", - "title": "Why is the sky blue? | National Geographic Kids", - "url": "https://kids.nationalgeographic.com/books/article/sky" - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.weather.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Weather Service (.gov)", - "snippet": "While all colors are scattered by air molecules, violet and blue are scattered most. The sky looks blue, not violet, because our e...", - "source_id": "2", - "title": "Why Is The Sky Blue? - National Weather Service", - "url": "https://www.weather.gov/fgz/SkyBlue" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.britannica.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Britannica", - "snippet": "One of the perennial questions of childhood is “Why is the sky blue?” You may have asked this as a child, or you may have a child ...", - "source_id": "9", - "title": "Why Is the Sky Blue? | Britannica", - "url": "https://www.britannica.com/story/why-is-the-sky-blue" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.uu.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Union University", - "snippet": "The sky appears blue because of light scattering. Light waves interact with air molecules, and the air particles quickly re-radiat...", - "source_id": "5", - "title": "Why is the sky blue on Earth, but black in space or on the Moon? | Science Guys | Union University, a Christian College in Tennessee", - "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://plus.nasa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "NASA+ (.gov)", - "snippet": "When sunlight enters Earth's atmosphere, it encounters gases that scatter the light in all directions. The shorter, choppier waves...", - "source_id": "4", - "title": "Space Place in a Snap: Why Is the Sky Blue?", - "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/#:~:text=When%20sunlight%20enters%20Earth's%20atmosphere%2C%20it%20encounters,sky%20appears%20blue%20on%20a%20sunny%20day." - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://morgridge.org&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Morgridge Institute for Research -", - "snippet": "The sky appears blue because of the scattering of blue light by the oxygen and nitrogen in our atmosphere. Light travels in waves,", - "source_id": "7", - "title": "Why is the sky blue? - Morgridge Institute for Research", - "url": "https://morgridge.org/blue-sky/why-is-the-sky-blue/" - } - ], - "type": "ai_overview" - }, - "error": null, - "section": "main", - "serp_rank": 0, - "sub_rank": 0, - "sub_type": "flat", - "text": "The sky appears blue due to a phenomenon called Rayleigh scattering . Sunlight reaches Earth's atmosphere and is scattered in all directions by gases and particles in the air. Because blue light travels in shorter, smaller waves, it is scattered more strongly than other colors, making the sky appear blue to our eyes. Rayleigh Scattering: As sunlight (which contains all colors of the rainbow) passes through the atmosphere, it collides with oxygen and nitrogen molecules. Wavelength Matters: Short, choppy wavelengths—blue and violet—are scattered about four times more than longer, redder wavelengths. Why Not Violet? While violet light has an even shorter wavelength than blue, the sky appears blue because the sun emits more blue light than violet, and human eyes are more sensitive to blue light . Atmosphere vs. Space: The atmosphere acts as a filter; without it, as seen from the Moon, the sky appears black. Horizon Effects: Near the horizon, the sky appears lighter or white because the scattered blue light has to pass through more air, causing it to scatter away again before reaching the eye. Sunsets: During sunset, the sun is low on the horizon, and light must pass through much more of the atmosphere. The blue light is scattered away completely, leaving red and yellow light to pass through.", - "title": null, - "type": "ai_overview", - "url": null - }, - { - "cite": null, - "cmpt_rank": 1, - "details": { - "items": [ - "Why is the sky blue short answer?", - "What is the true color of our sky?", - "How to explain to a kid why the sky is blue?", - "Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar" - ], - "type": "text" - }, - "error": null, - "section": "main", - "serp_rank": 1, - "sub_rank": 0, - "sub_type": null, - "text": "Why is the sky blue short answer?<|>What is the true color of our sky?<|>How to explain to a kid why the sky is blue?<|>Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar", - "title": null, - "type": "people_also_ask", - "url": null - }, - { - "cite": "https://spaceplace.nasa.gov › blue-sky", - "cmpt_rank": 2, - "details": null, - "error": null, - "section": "main", - "serp_rank": 2, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://spaceplace.nasa.gov/blue-sky/" - }, - { - "cite": "160+ comments · 2 years ago", - "cmpt_rank": 3, - "details": { - "items": [ - { - "text": "Eli5: why is the sky blue? : r/explainlikeimfive - Reddit", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1iytxes/eli5_why_is_the_sky_blue/" - }, - { - "text": "Why is the sky blue? Refraction. Why? Because ...", - "url": "https://www.reddit.com/r/askscience/comments/5tg0y2/why_is_the_sky_blue_refraction_why_because/" - }, - { - "text": "More results from www.reddit.com", - "url": "/search?q=why+is+the+sky+blue?+site:www.reddit.com&sca_esv=89aaa52b873f06b6&sa=X&ved=2ahUKEwiSt-LkwMOSAxXXm2oFHXpFH9kQrQIoAnoECCAQAw" - } - ], - "type": "hyperlinks" - }, - "error": null, - "section": "main", - "serp_rank": 3, - "sub_rank": 0, - "sub_type": "submenu", - "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", - "title": "Why is the sky blue? Do I understand it correctly", - "type": "general", - "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" - }, - { - "cite": "https://kids.nationalgeographic.com › books › article", - "cmpt_rank": 4, - "details": null, - "error": null, - "section": "main", - "serp_rank": 4, - "sub_rank": 0, - "sub_type": null, - "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today. Read more", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://kids.nationalgeographic.com/books/article/sky" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 5, - "sub_rank": 0, - "sub_type": "what_people_are_saying", - "text": null, - "title": "The REAL Reason the Sky Looks Blue", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 6, - "sub_rank": 1, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Sky Blue? Credit to Discover.", - "type": "perspectives", - "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 7, - "sub_rank": 2, - "sub_type": "what_people_are_saying", - "text": null, - "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", - "type": "perspectives", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 8, - "sub_rank": 3, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Sky Blue?", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/n4ElQOE3lyI" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 9, - "sub_rank": 4, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why Is the Sky Blue? Science Behind the Blue Sky\n\n#WhyIsTheSkyBlue\n#BlueSky\n#ScienceExplained\n#RayleighScattering\n#spacescience", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DT7ufKNj70z/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 10, - "sub_rank": 5, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", - "type": "perspectives", - "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 11, - "sub_rank": 6, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Night Sky Black and not Blue? | February 2 - February 8 | Star Gazers", - "type": "perspectives", - "url": "https://www.youtube.com/watch?v=pIZBeINyK8w" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 12, - "sub_rank": 7, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Did you know that the sky in GTA V is blue because it is a reference to the real life sky which is also blue", - "type": "perspectives", - "url": "https://www.reddit.com/r/GTAV/comments/1qtseyg/did_you_know_that_the_sky_in_gta_v_is_blue/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 13, - "sub_rank": 8, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 14, - "sub_rank": 9, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why the sky is blue", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DSOGSiPkTjw/?hl=en" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 15, - "sub_rank": 10, - "sub_type": "what_people_are_saying", - "text": null, - "title": "On a clear day, what is the colour of the sky?", - "type": "perspectives", - "url": "https://www.facebook.com/apna99036/posts/on-a-clear-day-what-is-the-colour-of-the-sky/1504710364995479/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 16, - "sub_rank": 11, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/o1pTCw8cg10" - }, - { - "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", - "cmpt_rank": 6, - "details": null, - "error": null, - "section": "main", - "serp_rank": 17, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", - "title": "Why Is the Sky Blue? - NESDIS - NOAA", - "type": "general", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" - }, - { - "cite": "https://math.ucr.edu › home › baez › physics › General", - "cmpt_rank": 7, - "details": null, - "error": null, - "section": "main", - "serp_rank": 18, - "sub_rank": 0, - "sub_type": null, - "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" - }, - { - "cite": "864K+ views · 5 years ago", - "cmpt_rank": 8, - "details": null, - "error": null, - "section": "main", - "serp_rank": 19, - "sub_rank": 0, - "sub_type": "video", - "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" - }, - { - "cite": "50+ reactions · 1 year ago", - "cmpt_rank": 9, - "details": null, - "error": null, - "section": "main", - "serp_rank": 20, - "sub_rank": 0, - "sub_type": "video", - "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", - "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", - "type": "general", - "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" - }, - { - "cite": "https://www.britannica.com › ... › Matter & Energy", - "cmpt_rank": 10, - "details": null, - "error": null, - "section": "main", - "serp_rank": 21, - "sub_rank": 0, - "sub_type": null, - "text": "Jan 22, 2026 — The midday sky appears blue, rather than a combination of blue and violet, because our eyes are more sensitive to blue light than to violet light. Read more", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://www.britannica.com/story/why-is-the-sky-blue" - }, - { - "cite": "https://www.uu.edu › dept › physics › scienceguys", - "cmpt_rank": 11, - "details": null, - "error": null, - "section": "main", - "serp_rank": 22, - "sub_rank": 0, - "sub_type": null, - "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", - "title": "Why is the sky blue on Earth, but black in space or ...", - "type": "general", - "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" - }, - { - "cite": null, - "cmpt_rank": 12, - "details": { - "heading": "People also search for", - "items": [ - "Why is the sky blue Class 10", - "Why is the sky blue for kids", - "Why is the sky blue no ai", - "Why is the sky blue reflection of ocean", - "Why is the sky blue and not violet", - "Why is the sky blue Army", - "Why is the sky blue long answer", - "Why is the sky blue essay" - ], - "type": "text" - }, - "error": null, - "section": "footer", - "serp_rank": 23, - "sub_rank": 0, - "sub_type": "people_also_search_for", - "text": "Why is the sky blue Class 10<|>Why is the sky blue for kids<|>Why is the sky blue no ai<|>Why is the sky blue reflection of ocean<|>Why is the sky blue and not violet<|>Why is the sky blue Army<|>Why is the sky blue long answer<|>Why is the sky blue essay", - "title": null, - "type": "searches_related", - "url": null - } - ] -} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[4f4d0fed0592].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[4f4d0fed0592].json new file mode 100644 index 00000000..eef04b88 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[4f4d0fed0592].json @@ -0,0 +1,300 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 770000000.0, + "result_estimate_time": 0.34 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "items": [ + { + "text": "Judgment|Law", + "url": "/search?sca_esv=52b056e1a97f92b3&q=Judgment&si=AKbGX_pf7wM0EvdxwkSTNU4_IH1iVsST_20-1VQ_gkawwK7Qt2EsmhrWPzqVxm-2NHTuj6c_USIEtpHCWa2QEWzlE7rNWplpJYSs4kGMy9-h1-TC4F3VuUyaY8a3esQlGPKttPqZpcXuECZM6Kk0TdZtUAGdTS3GqLzNRAgaG1sjLnN2qc-Wahk%3D&sa=X&ved=2ahUKEwi9qenRhfCEAxVgFVkFHR5nCXMQ3LoBegQICRAB" + } + ], + "type": "hyperlinks" + }, + "error": null, + "section": "header", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": null, + "type": "top_image_carousel", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "heading": "Choose what you’re giving feedback on", + "img_url": null, + "text": "the ability to make considered decisions or come to sensible conclusions. \"an error of judgment\" | a misfortune or calamity viewed as a divine punishment. \"the crash had been a judgment on the parents for wickedness\"", + "type": "panel", + "urls": [ + { + "text": "Oxford Languages", + "url": "https://languages.oup.com/google-dictionary-en" + }, + { + "text": "Learn more", + "url": "https://support.google.com/websearch/answer/10106608?hl=en" + }, + { + "text": " | |Learn to pronounce| | ", + "url": "/search?sca_esv=52b056e1a97f92b3&q=how+to+pronounce+judgment&stick=H4sIAAAAAAAAAOMIfcRoxS3w8sc9YSnDSWtOXmPU5uINKMrPK81LzkwsyczPExLhYglJLcoV4pHi4uLIKk1Jz03NK7FiUWJKzeNZxCqZkV-uUJKvUADUkw_UlKoAUwIABZwbvFsAAAA&pron_lang=en&pron_country=us&sa=X&ved=2ahUKEwi9qenRhfCEAxVgFVkFHR5nCXMQ3eEDegQIHhAM" + }, + { + "text": "sensible", + "url": "/search?sca_esv=52b056e1a97f92b3&q=sensible&si=AKbGX_qNq0Y8zql7SxzZAf2-HTTOokRcr0yDsJWWvxbU7kw0UFjrfeaDx79fPbPmSClxDgHhbF4uQVuSKTIDhsnFb3KSLUF4v1z7zK3uMEyEodOdh9E7UaU%3D&expnd=1" + }, + { + "text": "misfortune", + "url": "/search?sca_esv=52b056e1a97f92b3&q=misfortune&si=AKbGX_pvY3MWP4azJI0Z_NruCLb8MFymB6kqUF9igxdwSRscXVQQMqw_fJQpLMur7s6AYnCs9gXvH20ihJKQSLRmnJKFKneTKg2f_V4EJRS9ooX41b9XOmc%3D&expnd=1" + }, + { + "text": "calamity", + "url": "/search?sca_esv=52b056e1a97f92b3&q=calamity&si=AKbGX_qNq0Y8zql7SxzZAf2-HTTOx7IBv5oPgM2G6L6MdB1ePncKtJFNO2bvoOByD4KGoGSkU2OkH2duX_ov0CCw1EkKFTfdUk8Oy3hDGbbHuu49YhO9LFU%3D&expnd=1" + }, + { + "text": "See more", + "url": "/search?sca_esv=52b056e1a97f92b3&q=judgement&si=AKbGX_rLPMdHnrrwkrRo4VZlSHiJGnfs1HsKOELSf7cANrmSmZ25KLcpMynb0HUNv-7T1GwBb9qJnF7lgic-HwxvlRTE90i8qXtCM3EY7_QJbvpeMn7mdIs%3D&expnd=1&sa=X&ved=2ahUKEwi9qenRhfCEAxVgFVkFHR5nCXMQ2v4IegUIHhCMAQ" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "dictionary", + "text": "the ability to make considered decisions or come to sensible conclusions. \"an error of judgment\" | a misfortune or calamity viewed as a divine punishment. \"the crash had been a judgment on the parents for wickedness\"", + "title": "judgment", + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 2, + "details": { + "items": [ + "What is the true meaning of Judgement?", + "What is difference between Judgement and judgment?", + "What are 3 types of Judgement?", + "What is meant by judging someone?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "What is the true meaning of Judgement?<|>What is difference between Judgement and judgment?<|>What are 3 types of Judgement?<|>What is meant by judging someone?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.merriam-webster.com › dictionary › judgm...", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "a · the process of forming an opinion or evaluation by discerning and comparing . careful judgment of the odds ; b · an opinion or estimate so formed. is not worth ...", + "title": "Judgment Definition & Meaning", + "type": "general", + "url": "https://www.merriam-webster.com/dictionary/judgment" + }, + { + "cite": "https://dictionary.cambridge.org › dictionary › judgment", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "the ability to make good decisions or to form sensible opinions about something : Careful judgement will be needed in deciding whether it is trade policy or ...", + "title": "JUDGMENT | definition in the Cambridge English Dictionary", + "type": "general", + "url": "https://dictionary.cambridge.org/us/dictionary/english/judgment" + }, + { + "cite": "https://www.dictionary.com › browse › judgement", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "judgement · the act of establishing a relation between two or more terms , esp as an affirmation or denial · the expression of such a relation.", + "title": "JUDGMENT Definition & Usage Examples", + "type": "general", + "url": "https://www.dictionary.com/browse/judgement" + }, + { + "cite": "https://www.vocabulary.com › dictionary › judgment", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "/ˈdʒədʒmɛnt/ · noun. the act of judging or assessing a person or situation or event · noun. the cognitive process of reaching a decision or drawing conclusions.", + "title": "Judgment - Definition, Meaning & Synonyms", + "type": "general", + "url": "https://www.vocabulary.com/dictionary/judgment" + }, + { + "cite": "https://www.vocabulary.com › dictionary › judgement", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "judgement Add to list Share · noun. the act of judging or assessing a person or situation or event · noun. the cognitive process of reaching a decision or ...", + "title": "Judgement - Definition, Meaning & Synonyms", + "type": "general", + "url": "https://www.vocabulary.com/dictionary/judgement" + }, + { + "cite": "https://en.wikipedia.org › wiki › Judgement", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "Judgement (or judgment) is the evaluation of given circumstances to make a decision . Judgement is also the ability to make considered decisions.", + "title": "Judgement", + "type": "general", + "url": "https://en.wikipedia.org/wiki/Judgement" + }, + { + "cite": "https://www.consumerfinance.gov › ask-cfpb › what-is-...", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Aug 24, 2022 — A judgment is a court order that is the decision in a lawsuit . If a judgment is entered against you, a debt collector will have stronger tools, ...", + "title": "What is a judgment?", + "type": "general", + "url": "https://www.consumerfinance.gov/ask-cfpb/what-is-a-judgment-en-1381/" + }, + { + "cite": "https://www.dictionary.com › browse › judgment", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "an act or instance of judging . · the ability to judge, make a decision, or form an opinion objectively, authoritatively, and wisely, especially in matters ...", + "title": "JUDGMENT Definition & Usage Examples", + "type": "general", + "url": "https://www.dictionary.com/browse/judgment" + }, + { + "cite": "https://www.britannica.com › dictionary › judgment", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "JUDGMENT meaning: 1 : an opinion or decision that is based on careful thought ; 2 : the act or process of forming an opinion or making a decision after ...", + "title": "Judgment Definition & Meaning | Britannica Dictionary", + "type": "general", + "url": "https://www.britannica.com/dictionary/judgment" + }, + { + "cite": "https://www.collinsdictionary.com › dictionary › judgm...", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "A judgment is an opinion that you have or express after thinking carefully about something . In your judgment, what has changed over the past few years?", + "title": "JUDGMENT definition in American English", + "type": "general", + "url": "https://www.collinsdictionary.com/us/dictionary/english/judgment" + }, + { + "cite": null, + "cmpt_rank": 13, + "details": { + "heading": "Related searches", + "items": [ + "Define judgement in a sentence", + "define judgemental", + "judgment or judgement", + "judgement synonym", + "judgment meaning in law", + "meaning of judgement in the bible", + "judgement meaning in hindi", + "how to pronounce judgement" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "Define judgement in a sentence<|>define judgemental<|>judgment or judgement<|>judgement synonym<|>judgment meaning in law<|>meaning of judgement in the bible<|>judgement meaning in hindi<|>how to pronounce judgement", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 14, + "details": { + "img_urls": [ + "/search?q=define+judgement&sca_esv=52b056e1a97f92b3&tbm=isch&source=iu&ictx=1&vet=1&fir=bC5fjvsMiM5BYM%252C1A4mN5oBupXTrM%252C%252Fg%252F1q6jcl40b%253B60xhDKf3CMIkdM%252CdIUUBVnsqG2QAM%252C_%253BqbQb_HRQqIFvmM%252CZ8VC--9Y2ZUS8M%252C_%253BzVDy9-6TEVnBbM%252CqNogqLdMpuiY1M%252C_%253ByF9U4BEFkNYcbM%252Cu6WgUpVUXsDRYM%252C_&usg=AI4_-kRgJH4tl_ka8Z2kSO9u8cZs3hjOew&sa=X&ved=2ahUKEwi9qenRhfCEAxVgFVkFHR5nCXMQ_B16BAgiEAE#imgrc=bC5fjvsMiM5BYM", + "/search?q=define+judgement&sca_esv=52b056e1a97f92b3&tbm=isch&source=iu&ictx=1&vet=1&fir=bC5fjvsMiM5BYM%252C1A4mN5oBupXTrM%252C_%253B60xhDKf3CMIkdM%252CdIUUBVnsqG2QAM%252C_%253BqbQb_HRQqIFvmM%252CZ8VC--9Y2ZUS8M%252C_%253BzVDy9-6TEVnBbM%252CqNogqLdMpuiY1M%252C_%253ByF9U4BEFkNYcbM%252Cu6WgUpVUXsDRYM%252C_&usg=AI4_-kQo5jv4sAoJj-DWVRXYcjU_1XVZvQ&sa=X&ved=2ahUKEwi9qenRhfCEAxVgFVkFHR5nCXMQ_h16BAglEAE#imgrc=60xhDKf3CMIkdM", + "/search?q=define+judgement&sca_esv=52b056e1a97f92b3&tbm=isch&source=iu&ictx=1&vet=1&fir=bC5fjvsMiM5BYM%252C1A4mN5oBupXTrM%252C_%253B60xhDKf3CMIkdM%252CdIUUBVnsqG2QAM%252C_%253BqbQb_HRQqIFvmM%252CZ8VC--9Y2ZUS8M%252C_%253BzVDy9-6TEVnBbM%252CqNogqLdMpuiY1M%252C_%253ByF9U4BEFkNYcbM%252Cu6WgUpVUXsDRYM%252C_&usg=AI4_-kQo5jv4sAoJj-DWVRXYcjU_1XVZvQ&sa=X&ved=2ahUKEwi9qenRhfCEAxVgFVkFHR5nCXMQ_h16BAgmEAE#imgrc=qbQb_HRQqIFvmM", + "/search?q=define+judgement&sca_esv=52b056e1a97f92b3&tbm=isch&source=iu&ictx=1&vet=1&fir=bC5fjvsMiM5BYM%252C1A4mN5oBupXTrM%252C_%253B60xhDKf3CMIkdM%252CdIUUBVnsqG2QAM%252C_%253BqbQb_HRQqIFvmM%252CZ8VC--9Y2ZUS8M%252C_%253BzVDy9-6TEVnBbM%252CqNogqLdMpuiY1M%252C_%253ByF9U4BEFkNYcbM%252Cu6WgUpVUXsDRYM%252C_&usg=AI4_-kQo5jv4sAoJj-DWVRXYcjU_1XVZvQ&sa=X&ved=2ahUKEwi9qenRhfCEAxVgFVkFHR5nCXMQ_h16BAgnEAE#imgrc=zVDy9-6TEVnBbM", + "/search?q=define+judgement&sca_esv=52b056e1a97f92b3&tbm=isch&source=iu&ictx=1&vet=1&fir=bC5fjvsMiM5BYM%252C1A4mN5oBupXTrM%252C_%253B60xhDKf3CMIkdM%252CdIUUBVnsqG2QAM%252C_%253BqbQb_HRQqIFvmM%252CZ8VC--9Y2ZUS8M%252C_%253BzVDy9-6TEVnBbM%252CqNogqLdMpuiY1M%252C_%253ByF9U4BEFkNYcbM%252Cu6WgUpVUXsDRYM%252C_&usg=AI4_-kQo5jv4sAoJj-DWVRXYcjU_1XVZvQ&sa=X&ved=2ahUKEwi9qenRhfCEAxVgFVkFHR5nCXMQ_h16BAgjEAE#imgrc=yF9U4BEFkNYcbM", + "/search?sca_esv=52b056e1a97f92b3&q=define+judgement&tbm=isch&source=univ&fir=bC5fjvsMiM5BYM%252C1A4mN5oBupXTrM%252C_%253B60xhDKf3CMIkdM%252CdIUUBVnsqG2QAM%252C_%253BqbQb_HRQqIFvmM%252CZ8VC--9Y2ZUS8M%252C_%253BzVDy9-6TEVnBbM%252CqNogqLdMpuiY1M%252C_%253ByF9U4BEFkNYcbM%252Cu6WgUpVUXsDRYM%252C_&usg=AI4_-kQo5jv4sAoJj-DWVRXYcjU_1XVZvQ&sa=X&ved=2ahUKEwi9qenRhfCEAxVgFVkFHR5nCXMQiR56BAgkEAI" + ], + "type": "panel" + }, + "error": null, + "section": "rhs", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": "panel_rhs", + "text": "Judgement is the evaluation of given circumstances to make a decision. Judgement is also the ability to make considered decisions. The term has at least five distinct uses.", + "title": "Judgement", + "type": "knowledge", + "url": "https://en.wikipedia.org/wiki/Judgement" + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[513d1bfb479f].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[513d1bfb479f].json new file mode 100644 index 00000000..e4c7262e --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[513d1bfb479f].json @@ -0,0 +1,663 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 97500000.0, + "result_estimate_time": 0.4 + }, + "results": [ + { + "cite": "Farmers' Almanac", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "medium", + "text": null, + "title": "Why A Cake? Why Candles? The History of Birthdays Farmers' Almanac", + "type": "images", + "url": "https://www.farmersalmanac.com/why-a-birthday-cake-why-candles-2917" + }, + { + "cite": "Adobe Stock", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 1, + "sub_type": "medium", + "text": null, + "title": "Birthday Cake With Candles Images – Browse 381,323 Stock ... Adobe Stock", + "type": "images", + "url": "https://stock.adobe.com/search?k=birthday+cake+with+candles" + }, + { + "cite": "Anges de Sucre", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 2, + "sub_type": "medium", + "text": null, + "title": "Why do we put candles on a birthday cake? Anges de Sucre", + "type": "images", + "url": "https://www.angesdesucre.com/blogs/anges-de-sucre/why-do-we-put-candles-on-a-birthday-cake" + }, + { + "cite": "Daily Meal", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 3, + "sub_type": "medium", + "text": null, + "title": "Birthday Cake Candles Used To Signify Something Way Different Daily Meal", + "type": "images", + "url": "https://www.thedailymeal.com/1210564/birthday-cake-candles-used-to-signify-something-way-different/" + }, + { + "cite": "Adobe Stock", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 4, + "sub_type": "medium", + "text": null, + "title": "Birthday Cake With Candles Images – Browse 381,323 Stock ... Adobe Stock", + "type": "images", + "url": "https://stock.adobe.com/search?k=birthday+cake+with+candles" + }, + { + "cite": "TOPS Malibu", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 5, + "sub_type": "medium", + "text": null, + "title": "Premium Birthday Sparkler Candles (10-Pack) - TOPS Malibu TOPS Malibu", + "type": "images", + "url": "https://topsmalibu.com/products/mini-birthday-sparkler-candles-asst-color" + }, + { + "cite": "Unsplash", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 6, + "sub_type": "medium", + "text": null, + "title": "550+ Birthday Cake Candles Pictures | Download Free Images ... Unsplash", + "type": "images", + "url": "https://unsplash.com/s/photos/birthday-cake-candles" + }, + { + "cite": "Epicurious", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 7, + "sub_type": "medium", + "text": null, + "title": "39 Cute Birthday Candles for People Who Like Pretty Things ... Epicurious", + "type": "images", + "url": "https://www.epicurious.com/shopping/39-cute-birthday-candles-you-can-order-online" + }, + { + "cite": "Etsy", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 8, + "sub_type": "medium", + "text": null, + "title": "Pastel Happy Birthday Candles, Birthday Cake Candles, Pastel Birthday Cake Topper, Pastel Party Supplies, Birthday Cake Decor, Party Candles Etsy", + "type": "images", + "url": "https://www.etsy.com/listing/1045408407/pastel-happy-birthday-candles-birthday" + }, + { + "cite": "https://stock.adobe.com › search › k=birthday cake with ...", + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Search from thousands of royalty-free Birthday Cake With Candles stock images and video for your next project. Download royalty-free stock photos, vectors, ...", + "title": "Birthday Cake With Candles Images", + "type": "general", + "url": "https://stock.adobe.com/search?k=birthday%20cake%20with%20candles" + }, + { + "cite": "510+ followers", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Cakes With Candles ; the prettiest birthday cake ever. · Beautiful Birthday Cakes · Beautiful Cakes · Amazing Cakes · Happy Birthday Cakes For Women ; Love, Cake & ...", + "title": "120 Cakes With Candles ideas", + "type": "general", + "url": "https://www.pinterest.com/counterfitcakes/cakes-with-candles/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": "shopping", + "text": "Shop deals", + "title": "Save with deals on apparel, electronics, and more", + "type": "promo", + "url": "/search?sca_esv=52b056e1a97f92b3&q=shop+deals&sa=X&ved=2ahUKEwif_InXifCEAxXfFVkFHTk3B6sQ_hF6BAgIEAA" + }, + { + "cite": "https://unsplash.com › photos › birthday-cake-candles", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "Download the perfect birthday cake candles pictures. Find over 100+ of the best free birthday cake candles images. Free for commercial use ✓ No attribution ...", + "title": "550+ Birthday Cake Candles Pictures", + "type": "general", + "url": "https://unsplash.com/s/photos/birthday-cake-candles" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": { + "items": [ + "What is the significance of candles on a birthday cake?", + "How many candles should I put on my birthday cake?", + "How do you put a candle on a birthday cake?", + "What is the tradition of blowing out candles on a birthday cake?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": null, + "text": "What is the significance of candles on a birthday cake?<|>How many candles should I put on my birthday cake?<|>How do you put a candle on a birthday cake?<|>What is the tradition of blowing out candles on a birthday cake?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.gynzy.com › en-us › library › items › birth...", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": null, + "text": "Blow out the candles and pop all the balloons! Build the cake . Enter the student's name, choose the number of candles , and pick a song. Now comes the fun ...", + "title": "Birthday Cake", + "type": "general", + "url": "https://www.gynzy.com/en-us/library/items/birthday-cake" + }, + { + "cite": "Brant Florist", + "cmpt_rank": 7, + "details": { + "price": "$130.00", + "source": "Brant Florist", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Birthday Flower Cake with candles.", + "type": "products", + "url": null + }, + { + "cite": "picasasicilia.com", + "cmpt_rank": 7, + "details": { + "price": "$65.56", + "source": "picasasicilia.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "cake with candles.", + "type": "products", + "url": null + }, + { + "cite": "wuollet.com", + "cmpt_rank": 7, + "details": { + "price": "$45.00", + "source": "wuollet.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Flowers / Balloons / Candles Round Cake.", + "type": "products", + "url": null + }, + { + "cite": "Wolferman's Bakery", + "cmpt_rank": 7, + "details": { + "price": "$29.99", + "source": "Wolferman's Bakery", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Birthday Streusel Cake with Candles Gift, Pastries, Baked Goods Size Full by Wolfermans.", + "type": "products", + "url": null + }, + { + "cite": "classygirlcupcakes.com", + "cmpt_rank": 7, + "details": { + "price": "$95.00", + "source": "classygirlcupcakes.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Candle Layer Cake.", + "type": "products", + "url": null + }, + { + "cite": "Brant Florist", + "cmpt_rank": 7, + "details": { + "price": "$110.00", + "source": "Brant Florist", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Your Special Day Birthday Flower Cake with Candles.", + "type": "products", + "url": null + }, + { + "cite": "bonnieschicboutique.com", + "cmpt_rank": 7, + "details": { + "price": "$13.95", + "source": "bonnieschicboutique.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "NF - A194 Birthday Cake with Candle.", + "type": "products", + "url": null + }, + { + "cite": "anyoccasionballoons.com", + "cmpt_rank": 7, + "details": { + "price": "$20.00", + "source": "anyoccasionballoons.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Birthday Cake with candles.", + "type": "products", + "url": null + }, + { + "cite": "https://tenor.com › search › animated-birthday-cake-wit...", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 0, + "sub_type": null, + "text": "With Tenor, maker of GIF Keyboard, add popular Animated Birthday Cake With Candles animated GIFs to your conversations. Share the best GIFs now >>>", + "title": "Animated Birthday Cake With Candles GIFs", + "type": "general", + "url": "https://tenor.com/search/animated-birthday-cake-with-candles-gifs" + }, + { + "cite": "https://www.istockphoto.com › Stock photos", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 0, + "sub_type": null, + "text": "Browse 86,800+ birthday cake with lots of candles stock photos and images available, or search for cake with candles to find more great stock photos and ...", + "title": "Birthday Cake With Lots Of Candles Pictures, Images and ...", + "type": "general", + "url": "https://www.istockphoto.com/photos/birthday-cake-with-lots-of-candles" + }, + { + "cite": null, + "cmpt_rank": 10, + "details": { + "duration": "4 hr", + "ingredients": "Coconut cream, candle, blue, double cream, white sugar paste", + "source": "Eat Smarter", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": "Girls Birthday Cake with Candles", + "type": "recipes", + "url": "https://eatsmarter.com/recipes/girls-birthday-cake-with-candles" + }, + { + "cite": null, + "cmpt_rank": 10, + "details": { + "ingredients": "Meringue powder, black cocoa powder, unsweetened cocoa powder, powdered sugar, cinnamon", + "n_reviews": "1", + "rating": "5.0", + "source": "Ghoul at Heart", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 1, + "sub_type": null, + "text": null, + "title": "Noir Candle Cakes", + "type": "recipes", + "url": "https://ghoulatheart.com/noir-candle-cakes/" + }, + { + "cite": null, + "cmpt_rank": 10, + "details": { + "duration": "50 min", + "ingredients": "Layer cake, unsweetened cocoa powder, heavy cream, rich chocolate ovaltine, powdered sugar", + "n_reviews": "11", + "rating": "4.9", + "source": "Joy the Baker", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 2, + "sub_type": null, + "text": null, + "title": "Everybody's Birthday Cake", + "type": "recipes", + "url": "https://joythebaker.com/2016/01/everybodys-birthday-cake/" + }, + { + "cite": "https://www.quora.com › What-is-the-significance-of-bi...", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 0, + "sub_type": null, + "text": "May 1, 2023 — Putting candles on a cake was a special way to pay tribute to the Greek moon goddess, Artemis. They baked round cakes to symbolize the moon.", + "title": "What is the significance of birthday cake and candles ...", + "type": "general", + "url": "https://www.quora.com/What-is-the-significance-of-birthday-cake-and-candles-What-is-the-origin-of-the-tradition-of-blowing-out-candles-on-the-cake-during-birthdays" + }, + { + "cite": "https://www.shutterstock.com › search › birthday-cake-...", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 0, + "sub_type": null, + "text": "Birthday Cake Candles royalty-free images. 271,854 birthday cake candles stock photos, 3D objects, vectors, and illustrations are available royalty-free.", + "title": "Birthday cake with candles royalty-free images", + "type": "general", + "url": "https://www.shutterstock.com/search/birthday-cake-candles" + }, + { + "cite": "limogesbox.com", + "cmpt_rank": 13, + "details": { + "price": "$189.00", + "source": "limogesbox.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Birthday Cake with Candle.", + "type": "products", + "url": null + }, + { + "cite": "flowerandvinesedalia.com", + "cmpt_rank": 13, + "details": { + "price": "$59.00", + "source": "flowerandvinesedalia.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Birthday Cake.", + "type": "products", + "url": null + }, + { + "cite": "Silhouette Design Store", + "cmpt_rank": 13, + "details": { + "price": "$0.99", + "source": "Silhouette Design Store", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "birthday cake with candles.", + "type": "products", + "url": null + }, + { + "cite": "eBay - everydayideas", + "cmpt_rank": 13, + "details": { + "price": "$185.32", + "source": "eBay - everydayideas", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 33, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Fraser Hill Farm 6-ft. Tall Happy Birthday Cake With 4 Faux Candles,.", + "type": "products", + "url": null + }, + { + "cite": "Freed's Bakery", + "cmpt_rank": 13, + "details": { + "price": "$128.95", + "source": "Freed's Bakery", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 34, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Comic Candle Icing Cake 7\" Round (Serves 8-10) / Lemon Cake w/ Lemon Mousse.", + "type": "products", + "url": null + }, + { + "cite": "www.limogesboutique.com", + "cmpt_rank": 13, + "details": { + "price": "$265.00", + "source": "www.limogesboutique.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 35, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Birthday Cake With Candle.", + "type": "products", + "url": null + }, + { + "cite": "ladentelliere.com", + "cmpt_rank": 13, + "details": { + "price": "$239.00", + "source": "ladentelliere.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 36, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Birthday Cake W/Pink Candle.", + "type": "products", + "url": null + }, + { + "cite": "Balloon Kings", + "cmpt_rank": 13, + "details": { + "price": "$18.00", + "source": "Balloon Kings", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 37, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Birthday Cake & Candles.", + "type": "products", + "url": null + }, + { + "cite": null, + "cmpt_rank": 14, + "details": { + "heading": "Related searches", + "items": [ + "birthday cake with candles gif", + "Birthday cake with candles for girl", + "birthday cake with candles images", + "birthday cake with candles drawing", + "birthday cake with candles clipart", + "Birthday cake with candles for adults" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 38, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "birthday cake with candles gif<|>Birthday cake with candles for girl<|>birthday cake with candles images<|>birthday cake with candles drawing<|>birthday cake with candles clipart<|>Birthday cake with candles for adults", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[760310a7eec2].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[760310a7eec2].json new file mode 100644 index 00000000..38e76b8f --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[760310a7eec2].json @@ -0,0 +1,763 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": null, + "result_estimate_time": null + }, + "results": [ + { + "cite": "Pinterest", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "medium", + "text": null, + "title": "future closet | Old school fashion, 80s fashion men, Hipster ... Pinterest", + "type": "images", + "url": "https://www.pinterest.com/pin/future-closet--155866837095395538/" + }, + { + "cite": "Vintage Dancer", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 1, + "sub_type": "medium", + "text": null, + "title": "Vintage Men's Outfits -1920s, 1930s, 1940s, 1950s Vintage Dancer", + "type": "images", + "url": "https://vintagedancer.com/vintage/casual-vintage-inspired-mens-styles/" + }, + { + "cite": "GQ", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 2, + "sub_type": "medium", + "text": null, + "title": "Best Vintage Clothing for Men in 2021: 52 Classic Menswear ... GQ", + "type": "images", + "url": "https://www.gq.com/story/best-vintage-menswear" + }, + { + "cite": "210+ followers", + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Apr 21, 2014 - Explore Orquidea Feliz de Gomez's board \" Old sChoOl men's fashion \" on Pinterest. See more ideas about mens fashion , mens outfits , ...", + "title": "49 Old sChoOl men's fashion ideas", + "type": "general", + "url": "https://www.pinterest.com/felizdegomez/old-school-mens-fashion/" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": { + "items": [ + "How to dress like an old school guy?", + "How to look like a vintage man?", + "What is retro theme for men?", + "How does a 70 year old man dress?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "How to dress like an old school guy?<|>How to look like a vintage man?<|>What is retro theme for men?<|>How does a 70 year old man dress?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "880+ followers", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": "image_strip", + "text": "Jun 21, 2022 - Explore Jimmie Burnett's board \" old school \", followed by 282 people on Pinterest. See more ideas about mens outfits , mens fashion , ...", + "title": "Old school - mens outfits, mens fashion, well dressed men", + "type": "general", + "url": "https://www.pinterest.com/burnett1917/old-school/" + }, + { + "cite": "www.reddit.com  ·  Sep 18, 2022", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": "Advice for young men with more old fashioned tastes : r/malefashionadvice - Reddit", + "type": "discussions_and_forums", + "url": "https://www.reddit.com/r/malefashionadvice/comments/xh23sq/advice_for_young_men_with_more_old_fashioned/" + }, + { + "cite": "www.reddit.com  ·  Mar 19, 2016  ·  38 posts", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 1, + "sub_type": null, + "text": null, + "title": "What old timey fashion trends for men need to make a comeback? : r/AskMen - Reddit", + "type": "discussions_and_forums", + "url": "https://www.reddit.com/r/AskMen/comments/4b1h1n/what_old_timey_fashion_trends_for_men_need_to/" + }, + { + "cite": "askandyaboutclothes.com  ·  Feb 2, 2006", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 2, + "sub_type": null, + "text": null, + "title": "Old School Tie - Forums - Ask Andy About Clothes", + "type": "discussions_and_forums", + "url": "https://askandyaboutclothes.com/community/threads/old-school-tie.46678/" + }, + { + "cite": "https://www.thrifted.com › pages › mens-vintage-home...", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Shop the mens vintage clothing range at Thrifted. 1000s of handpicked retro & vintage items to choose from. Designer vintage, retro sportswear, vintage ...", + "title": "Mens Vintage Clothing | The Online Vintage Store", + "type": "general", + "url": "https://www.thrifted.com/pages/mens-vintage-homepage" + }, + { + "cite": "120+ comments · 1 year ago", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "The simplest way to adapt your usual style would be to get a few OCBDs to hem and wear untucked. You should also look into chambray, linen, or ...", + "title": "Advice for young men with more old fashioned tastes", + "type": "general", + "url": "https://www.reddit.com/r/malefashionadvice/comments/xh23sq/advice_for_young_men_with_more_old_fashioned/" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": { + "n_reviews": "6.3K", + "rating": "4.6", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": "brands", + "text": "Each selection of button-down and snap-front men's retro shirts may have \nsubtle features like a sawtooth front pocket or two-point back yoke or a \nrange of horn ...", + "title": "Wrangler", + "type": "products", + "url": "https://www.wrangler.com/shop/men-featured-collections-retro" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": { + "n_reviews": "40", + "rating": "3.2", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 1, + "sub_type": "brands", + "text": "9090 products — SLAM Vintage Tee NEW JERSEY NETS · View Details. Select a Size. Small. \nMedium. Large. X Large. 2X Large. 3X ...", + "title": "Mitchell & Ness", + "type": "products", + "url": "https://www.mitchellandness.com/men/mens-clothing" + }, + { + "cite": null, + "cmpt_rank": 7, + "details": { + "n_reviews": "91", + "rating": "4.6", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 2, + "sub_type": "brands", + "text": "Introducing Levi's® Vintage Clothing for men at Levi.com. Browse our \nhistorically accurate recreations of iconic vintage men's clothing and \nstyles at ...", + "title": "Levi's", + "type": "products", + "url": "https://www.levi.com/US/en_US/levis-vintage-clothing/levis-vintage-clothing-for-men/c/levi_clothing_men_vintage_clothing_us" + }, + { + "cite": "https://www.etsy.com › market › old_school_men's_wear", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": "image_strip", + "text": "Check out our old school men's wear selection for the very best in unique or custom, handmade pieces from our t- shirts shops.", + "title": "Old School Men's Wear", + "type": "general", + "url": "https://www.etsy.com/market/old_school_men%27s_wear" + }, + { + "cite": "https://www.swankysvintage.com", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": null, + "text": "Welcome to our store, where you can find the finest vintage style clothing for your every need. Take a look at our collections and enjoy shopping online at ...", + "title": "Mens Vintage Clothing | www.swankysvintage.com | United ...", + "type": "general", + "url": "https://www.swankysvintage.com/" + }, + { + "cite": "https://www.historicalemporium.com › ... › Outfits", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 0, + "sub_type": "image_strip", + "text": "Mens Old West Costumes and Frontier Fashion We carry a full line of period-correct 1800s clothing , perfect for cowboy shooters, living history enthusiasts, ...", + "title": "Old West Outfits | Mens Outfits", + "type": "general", + "url": "https://www.historicalemporium.com/store/outfits.php?&category=Mens+Outfits&theme=Old+West" + }, + { + "cite": null, + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": "shopping", + "text": "Shop deals", + "title": "Save with deals on apparel, electronics, and more", + "type": "promo", + "url": "/search?sca_esv=52b056e1a97f92b3&q=shop+deals&sa=X&ved=2ahUKEwiO0IjMhvCEAxUfFVkFHb0zDygQ_hF6BAhTEAA" + }, + { + "cite": "Historical Emporium", + "cmpt_rank": 12, + "details": { + "n_reviews": "44", + "price": "$73.95", + "rating": "4.8", + "source": "Historical Emporium", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Mens Cotton Stripe Banker/Club Collar Work Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Penner's", + "cmpt_rank": 12, + "details": { + "price": "$78.50", + "source": "Penner's", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Penner's Men's El Clasico Retro Camp Shirt.", + "type": "products", + "url": null + }, + { + "cite": "eBay - designerbrandsforless", + "cmpt_rank": 12, + "details": { + "n_reviews": "13", + "price": "$25.88", + "rating": "3.2", + "source": "eBay - designerbrandsforless", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Maximos Men's Retro Classic Two Tone Bowling Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Historical Emporium", + "cmpt_rank": 12, + "details": { + "n_reviews": "18", + "price": "$66.95", + "rating": "4.9", + "source": "Historical Emporium", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Mens Cotton Band Collar Work Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Historical Emporium", + "cmpt_rank": 12, + "details": { + "price": "$66.95", + "source": "Historical Emporium", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Classic Mens Brown Cotton Stripe Band Collar Work Shirt (Size 2X) | 19th Century | Historical | Vintage | Antique.", + "type": "products", + "url": null + }, + { + "cite": "Walmart - G-Style USA", + "cmpt_rank": 12, + "details": { + "n_reviews": "1", + "price": "$29.95", + "rating": "4.0", + "source": "Walmart - G-Style USA", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "G-Style USA Men's Two Tone Retro Button Down Bowling Shirt.", + "type": "products", + "url": null + }, + { + "cite": "eBay - designerbrandsforless", + "cmpt_rank": 12, + "details": { + "price": "$25.88", + "source": "eBay - designerbrandsforless", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Maximos Men's Retro Classic Two Tone Bowling Casual Dress Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Kubosh Clothing Company", + "cmpt_rank": 12, + "details": { + "price": "$25.00", + "source": "Kubosh Clothing Company", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Old School Men’s premium heavyweight tee.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 13, + "details": { + "price": "$49.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Mepase Men's Retro 80s 90s Outfit.", + "type": "products", + "url": null + }, + { + "cite": "Historical Emporium", + "cmpt_rank": 13, + "details": { + "n_reviews": "18", + "price": "$66.95", + "rating": "4.9", + "source": "Historical Emporium", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Mens Cotton Band Collar Work Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Tractor Supply Company", + "cmpt_rank": 13, + "details": { + "n_reviews": "68", + "price": "$49.99", + "rating": "3.9", + "source": "Tractor Supply Company", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Wrangler Men's Retro Long Sleeve Western Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Royaura", + "cmpt_rank": 13, + "details": { + "price": "$23.39", + "source": "Royaura", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Charlie Sheen Men's Vintage 50s Hot Rod Cuban Collar Bowling Shirt. SALE.", + "type": "products", + "url": null + }, + { + "cite": "snapklik.com", + "cmpt_rank": 13, + "details": { + "price": "$39.98", + "source": "snapklik.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Men's Vintage 1950s Casual Short Sleeve Button Up Cuban Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 13, + "details": { + "price": "$32.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Men's 70s Disco Hip Hop Casual Long Sleeve Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Fearless Apparel", + "cmpt_rank": 13, + "details": { + "price": "$56.99", + "source": "Fearless Apparel", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Steady Clothing Men's Classy Piston V8 Bowling Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Historical Emporium", + "cmpt_rank": 13, + "details": { + "n_reviews": "44", + "price": "$73.95", + "rating": "4.8", + "source": "Historical Emporium", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 33, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Mens Cotton Stripe Banker/Club Collar Work Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Royaura", + "cmpt_rank": 14, + "details": { + "price": "$17.99", + "source": "Royaura", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 34, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Men's 50s Vintage Tony Bowling Shirt. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Wrangler", + "cmpt_rank": 14, + "details": { + "n_reviews": "11", + "price": "$59.99", + "rating": "4.9", + "source": "Wrangler", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 35, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Wrangler Men's Retro Premium Olive Solid Snap Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 14, + "details": { + "price": "$14.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 36, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Qzh.duao Mens 70s Disco Shirts Casual Paisley Tops Floral Vintage Long Sleeve Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Harmony Gallery", + "cmpt_rank": 14, + "details": { + "price": "$79.00", + "source": "Harmony Gallery", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 37, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "American Retro Workwear Men's Relaxed Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Trendy Hip Buys", + "cmpt_rank": 14, + "details": { + "price": "$44.00", + "source": "Trendy Hip Buys", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 38, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "70s Style Color Block Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Royaura", + "cmpt_rank": 14, + "details": { + "price": "$18.89", + "source": "Royaura", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 39, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Men's Vintage Short Sleeve Bowling Shirt. SALE.", + "type": "products", + "url": null + }, + { + "cite": "Wrangler", + "cmpt_rank": 14, + "details": { + "n_reviews": "15", + "price": "$59.99", + "rating": "4.7", + "source": "Wrangler", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 40, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Wrangler Men's Retro Premium Shirt.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 14, + "details": { + "price": "$32.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 41, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "DOOPCCOR Men's 80s 90s Retro Sweatpants.", + "type": "products", + "url": null + }, + { + "cite": null, + "cmpt_rank": 15, + "details": { + "heading": "Related searches", + "items": [ + "modern vintage outfit for male", + "vintage mens clothing 1970s", + "retro outfit for male 90s", + "old school dressing for females", + "old school dressing in the 70s", + "old school outfits 90s" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 42, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "modern vintage outfit for male<|>vintage mens clothing 1970s<|>retro outfit for male 90s<|>old school dressing for females<|>old school dressing in the 70s<|>old school outfits 90s", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[7d621e868493].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[7d621e868493].json new file mode 100644 index 00000000..6a2c414e --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[7d621e868493].json @@ -0,0 +1,404 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard-overview", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 9070000.0, + "result_estimate_time": 0.35 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": "Oscar the Grouch", + "img_url": null, + "text": "Oscar the Grouch|Muppet character|Muppet character|Overview|Overview|Overview|Videos|Videos|Videos|Movies|Movies|Movies|Songs|Songs|Songs|Played by|Played by|Played by|Quotes|Quotes|Quotes|Listen|Listen|Listen", + "type": "panel" + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "panel", + "text": null, + "title": "Oscar the Grouch Muppet character Overview Videos Movies Songs Played by Quotes Listen", + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "heading": null, + "img_url": null, + "type": "panel", + "urls": [ + { + "text": "", + "url": "/search?sca_esv=52b056e1a97f92b3&q=oscar+the+grouch&tbm=isch&source=univ&sa=X&ved=2ahUKEwjWqLHmivCEAxWHGFkFHVugAIYQnN8JegQIExAD" + }, + { + "text": "30:15|YouTube| • |Sesame Street|Sesame Street: Best of Oscar the Grouch Compilation | Happy ...|Celebrate Oscar's birthday with some of our favorite moments of our resident Grouch from classic songs like I Love Trash to new hits like If ...|May 27, 2022", + "url": "https://www.youtube.com/watch?v=FiZvEXqWGI0" + }, + { + "text": "", + "url": "https://twitter.com/OscarTheGrouch" + }, + { + "text": "YouTube| • |Sesame Street|2:30|Dec 18, 2009|Sesame Street: I Love Trash", + "url": "https://www.youtube.com/watch?v=rxgWHzMvXOY" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "featured_results", + "text": "30:15 YouTube • Sesame Street Sesame Street: Best of Oscar the Grouch Compilation | Happy ... Celebrate Oscar's birthday with some of our favorite moments of our resident Grouch from classic songs like I Love Trash to new hits like If ... May 27, 2022 X Oscar the Grouch (@OscarTheGrouch) / X YouTube • Sesame Street 2:30 Dec 18, 2009 Sesame Street: I Love Trash", + "title": null, + "type": "knowledge", + "url": "https://www.youtube.com/watch?v=FiZvEXqWGI0" + }, + { + "cite": "https://en.wikipedia.org › wiki › Oscar_the_Grouch", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Oscar the Grouch is a Muppet character created by Jim Henson and Jon Stone for the PBS/HBO children's television program Sesame Street. He has a green body, ...", + "title": "Oscar the Grouch", + "type": "general", + "url": "https://en.wikipedia.org/wiki/Oscar_the_Grouch" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": { + "items": [ + "Why did they remove Oscar the Grouch?", + "Why is Oscar the Grouch so grouchy?", + "Why did Oscar the Grouch go from orange to green?", + "What is the point of Oscar the Grouch?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Why did they remove Oscar the Grouch?<|>Why is Oscar the Grouch so grouchy?<|>Why did Oscar the Grouch go from orange to green?<|>What is the point of Oscar the Grouch?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://twitter.com/OscarTheGrouch", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": "header", + "text": null, + "title": "Oscar the Grouch (@OscarTheGrouch) · X X (Twitter) https://twitter.com/OscarTheGrouch", + "type": "twitter_cards", + "url": "https://twitter.com/OscarTheGrouch?ref_src=twsrc^google|twcamp^serp|twgr^author" + }, + { + "cite": "Posted on X · 2 days ago", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 1, + "sub_type": "card", + "text": "In my house, every night is Oscar night. #Oscar", + "title": "@OscarTheGrouch", + "type": "twitter_cards", + "url": "https://twitter.com/OscarTheGrouch/status/1766865706149130365?ref_src=twsrc^google|twcamp^serp|twgr^tweet" + }, + { + "cite": "Posted on X · 3 days ago", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 2, + "sub_type": "card", + "text": "With one less hour of sleep, everyone is going to be (insert your name) the Grouch! #DaylightSavingsTime", + "title": "@OscarTheGrouch", + "type": "twitter_cards", + "url": "https://twitter.com/OscarTheGrouch/status/1766813889805623687?ref_src=twsrc^google|twcamp^serp|twgr^tweet" + }, + { + "cite": "Posted on X · Mar 5, 2024", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 3, + "sub_type": "card", + "text": "I love trying everything only once, because without practice I’m terrible at it and that makes me grouchy! I love it! 🏓🎥🔬🎸🛹🎨🎮🪕🎤 ", + "title": "@OscarTheGrouch", + "type": "twitter_cards", + "url": "https://twitter.com/OscarTheGrouch/status/1765083198092935472?ref_src=twsrc^google|twcamp^serp|twgr^tweet" + }, + { + "cite": "https://muppet.fandom.com › wiki › Oscar_the_Grouch", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "Oscar the Grouch is a furry, green Grouch who lives in a trash can on Sesame Street. In fact, he loves trash so much that he's rarely seen outside of his ...", + "title": "Oscar the Grouch | Muppet Wiki - Fandom", + "type": "general", + "url": "https://muppet.fandom.com/wiki/Oscar_the_Grouch" + }, + { + "cite": "https://sesameworkshop.org › sesame-street-characters", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Oscar , Sesame Street's resident Grouch , despises all things “nice” like rainbows, candy, and chocolate. Being grouchy makes him happy, but being happy makes him ...", + "title": "Oscar the Grouch", + "type": "general", + "url": "https://sesameworkshop.org/our-work/shows/sesame-street/sesame-street-characters/oscar-the-grouch/" + }, + { + "cite": "YouTube", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": "unspecified-3", + "text": null, + "title": "Sesame Street: Best of Oscar the Grouch Compilation | Happy ... YouTube · Sesame Street 30 minutes, 15 seconds May 27, 2022", + "type": "videos", + "url": "https://www.youtube.com/watch?v=FiZvEXqWGI0" + }, + { + "cite": "YouTube", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 1, + "sub_type": "unspecified-3", + "text": null, + "title": "Sesame Street: I Love Trash YouTube · Sesame Street 2 minutes, 30 seconds Dec 18, 2009", + "type": "videos", + "url": "https://www.youtube.com/watch?v=rxgWHzMvXOY" + }, + { + "cite": "YouTube", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 2, + "sub_type": "unspecified-3", + "text": null, + "title": "Elmo the Grouch | Sesame Street Full Episode YouTube · Sesame Street 25 minutes, 57 seconds Mar 3, 2023", + "type": "videos", + "url": "https://www.youtube.com/watch?v=avyZ67cRZiY" + }, + { + "cite": "YouTube", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 3, + "sub_type": "unspecified-3", + "text": null, + "title": "Sesame Street: Oscar the Grouch Says #StayHome YouTube · Sesame Street 38 seconds Apr 6, 2020", + "type": "videos", + "url": "https://www.youtube.com/watch?v=0wGYi6o6qT0" + }, + { + "cite": "https://muppet.fandom.com › wiki › Oscar_the_Grouch_...", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": null, + "text": "A new Oscar is built with a more circular head and longer, shaggier fur. This variant first appeared in A Swingin' Sesame Street Celebration. During season 51, ...", + "title": "Oscar the Grouch Through the Years | Muppet Wiki - Fandom", + "type": "general", + "url": "https://muppet.fandom.com/wiki/Oscar_the_Grouch_Through_the_Years" + }, + { + "cite": "https://gund.com › products › oscar-the-grouch-10-in", + "cmpt_rank": 9, + "details": { + "price": "25.00", + "stock": " Free 3–7 day delivery over $50", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "Oscar features a friendly embroidered smile and outstretched arms, ready for a hug! This plush toy is surface-washable for easy cleaning and ships in a ...", + "title": "Oscar the Grouch, 10 in", + "type": "general", + "url": "https://gund.com/products/oscar-the-grouch-10-in" + }, + { + "cite": null, + "cmpt_rank": 10, + "details": { + "heading": "Movies", + "img_url": null, + "items": [ + "The Adventures of Elmo in Grouchland", + "Sesame Street Presents: Follow That Bird", + "Elmo Saves Christmas", + "Elmo's Christmas Countdown", + "A Muppet Family Christmas", + "The Great Muppet Caper", + "Night at the Museum: Battle of the Smithsonian" + ], + "type": "panel", + "urls": [] + }, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 0, + "sub_type": "movies", + "text": "The Adventures of Elmo in Grouchland", + "title": "Movies", + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 11, + "details": { + "heading": "Songs", + "img_url": null, + "type": "panel", + "urls": [] + }, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": "songs", + "text": null, + "title": "Songs", + "type": "knowledge", + "url": null + }, + { + "cite": "https://www.si.edu › oscar-grouch-puppet:nmah_1182904", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": null, + "text": "1970-1980. Oscar is a green furry monster who lives in a trash can on Sesame Street and frequently argues with the other Muppet and human characters. Despite ...", + "title": "Oscar the Grouch puppet", + "type": "general", + "url": "https://www.si.edu/object/oscar-grouch-puppet%3Anmah_1182904" + }, + { + "cite": null, + "cmpt_rank": 13, + "details": { + "heading": "Played by", + "img_url": null, + "items": [ + "Caroll Spinney", + "Eric Jacobson" + ], + "type": "panel", + "urls": [] + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": "played-by", + "text": "Caroll Spinney", + "title": "Played by", + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 14, + "details": { + "heading": "Related searches", + "items": [ + "why was oscar the grouch removed from sesame street", + "oscar the grouch puppet", + "oscar the grouch costume", + "oscar the grouch controversy", + "oscar the grouch girlfriend", + "oscar the grouch gif", + "why was oscar the grouch orange", + "oscar the grouch shirt" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "why was oscar the grouch removed from sesame street<|>oscar the grouch puppet<|>oscar the grouch costume<|>oscar the grouch controversy<|>oscar the grouch girlfriend<|>oscar the grouch gif<|>why was oscar the grouch orange<|>oscar the grouch shirt", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 15, + "details": null, + "error": null, + "section": "rhs", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": "panel_rhs", + "text": "Oscar the Grouch is a Muppet character created by Jim Henson and Jon Stone for the PBS/HBO children's television program Sesame Street. He has a green body, no visible nose, and lives in a trash can.", + "title": null, + "type": "knowledge", + "url": "https://en.wikipedia.org/wiki/Oscar_the_Grouch" + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[81f344a9684a].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[81f344a9684a].json new file mode 100644 index 00000000..756fe8d3 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[81f344a9684a].json @@ -0,0 +1,293 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 64200000.0, + "result_estimate_time": 0.3 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": "Choose what you’re giving feedback on", + "img_url": null, + "text": "a tank for storing water, especially one supplying taps or as part of a flushing toilet. | an underground reservoir for rainwater .", + "type": "panel", + "urls": [ + { + "text": "Oxford Languages", + "url": "https://languages.oup.com/google-dictionary-en" + }, + { + "text": "Learn more", + "url": "https://support.google.com/websearch/answer/10106608?hl=en" + }, + { + "text": " | |Learn to pronounce| | ", + "url": "/search?sca_esv=52b056e1a97f92b3&q=how+to+pronounce+cistern&stick=H4sIAAAAAAAAAOMIfcRoyS3w8sc9YSmDSWtOXmPU4uINKMrPK81LzkwsyczPExLmYglJLcoV4pbi5GJPziwuSS3Ks2JRYkrN41nEKpGRX65Qkq9QANSSD9STqgBVAQDb7UeKWQAAAA&pron_lang=en&pron_country=us&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQ3eEDegQIKhAM" + }, + { + "text": "supplying", + "url": "/search?sca_esv=52b056e1a97f92b3&q=supplying&si=AKbGX_onJk-q0LQUYzV7-GRhpJ5Dcp491JEo9xGGOogmfD72qMA2dnDhv-39t69a-nJTCQmkjJ8x3SdBFZ6796vpkHD7NXTHetnGv1fFRmGidhhQodOz6fo%3D&expnd=1" + }, + { + "text": "taps", + "url": "/search?sca_esv=52b056e1a97f92b3&q=taps&si=AKbGX_o1kf-QqTkl_gxFsV7N2y1VIxXxokBXTaDQ5VYe_CCESwFq_yyPrvQAvXLfP3ngj90Mauqo20IinPmLH8uupBh1kYgAXg%3D%3D&expnd=1" + }, + { + "text": "flushing", + "url": "/search?sca_esv=52b056e1a97f92b3&q=flushing&si=AKbGX_qNq0Y8zql7SxzZAf2-HTTOc2PQ9UUQ96IglJZNpqY7qqXQcYa9ke2xO0Fn8IriBAh-2tbyEDTkdb_WGqtYZ2VT-uX-O0Of22zI-UpIfOn9wNP1XFQ%3D&expnd=1" + }, + { + "text": "reservoir", + "url": "/search?sca_esv=52b056e1a97f92b3&q=reservoir&si=AKbGX_onJk-q0LQUYzV7-GRhpJ5DJGyZxhtUTrqO4JoQo76fyRUiLIxj7y-zR68oHMYOqyzeOoBk41ESwyuAClbH5EK0VioTCg4Dq2nXygN7zfzVw-WVbNI%3D&expnd=1" + }, + { + "text": "rainwater", + "url": "/search?sca_esv=52b056e1a97f92b3&q=rainwater&si=AKbGX_onJk-q0LQUYzV7-GRhpJ5DNOluQE4w-7_6PpCJqOf--sXPZNLqI1WMfjnpKWkwHwY6nhi4SPPpV7hpsLkNNRmhZtr0eEQRCGuuTX26rI340l5J8H0%3D&expnd=1" + }, + { + "text": "See more", + "url": "/search?sca_esv=52b056e1a97f92b3&q=cistern&si=AKbGX_r0zqXEeLlZhGfi3fbO0QSWKGCL9sqcT40iNLNZLVskkyPJ6exvr9sNypbPonRnw6tihnfDNZjwUuDGhJRG2yw-sAVGig%3D%3D&expnd=1&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQ2v4IegQIKhAj" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "dictionary", + "text": "a tank for storing water, especially one supplying taps or as part of a flushing toilet. | an underground reservoir for rainwater .", + "title": "cistern", + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "items": [ + "Why would a house have a cistern?", + "What was the purpose of a cistern?", + "What is the difference between a cistern and a well?", + "What is the difference between a cistern and a tank?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Why would a house have a cistern?<|>What was the purpose of a cistern?<|>What is the difference between a cistern and a well?<|>What is the difference between a cistern and a tank?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://en.wikipedia.org › wiki › Cistern", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "A cistern is a waterproof receptacle for holding liquids, usually water. Cisterns are often built to catch and store rainwater. Cisterns are distinguished ...", + "title": "Cistern", + "type": "general", + "url": "https://en.wikipedia.org/wiki/Cistern" + }, + { + "cite": "https://extension.psu.edu › rainwater-cisterns-design-cons...", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Dec 7, 2022 — Roof-catchment cisterns are systems used to collect and store rainwater for household and other uses. Save for later Print. Share. Updated:.", + "title": "Rainwater Cisterns: Design, Construction, and Treatment", + "type": "general", + "url": "https://extension.psu.edu/rainwater-cisterns-design-construction-and-treatment" + }, + { + "cite": "https://www.merriam-webster.com › dictionary › cistern", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Mar 3, 2024 — The meaning of CISTERN is an artificial reservoir (such as an underground tank) for storing liquids and especially water (such as ...", + "title": "Cistern Definition & Meaning", + "type": "general", + "url": "https://www.merriam-webster.com/dictionary/cistern" + }, + { + "cite": "https://www.vocabulary.com › dictionary › cistern", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "A cistern is an underground tank that holds water. A long time ago, cistern water was used for drinking, but nowadays developed parts of the world mainly use it ...", + "title": "Cistern - Definition, Meaning & Synonyms", + "type": "general", + "url": "https://www.vocabulary.com/dictionary/cistern" + }, + { + "cite": "https://www.esf.edu › ere › GICalculator › CisternsIntro", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "Cisterns are large tanks that store rainwater collected from impervious surfaces for domestic uses or for consumption. collapse section Why use Cisterns ?", + "title": "Cisterns", + "type": "general", + "url": "https://www.esf.edu/ere/endreny/GICalculator/CisternsIntro.html" + }, + { + "cite": "https://dictionary.cambridge.org › dictionary › cistern", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "CISTERN meaning: 1. a container in which water is stored, especially one connected to a toilet or in the roof of a…. Learn more.", + "title": "CISTERN | definition in the Cambridge English Dictionary", + "type": "general", + "url": "https://dictionary.cambridge.org/us/dictionary/english/cistern" + }, + { + "cite": "https://www.currenenvironmental.com › blog › what-is-a...", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "Feb 16, 2023 — A cistern is a buried tank, commonly made of brick & are typically associated with capturing and storing rainwater. Cisterns are not cess ...", + "title": "What is a Cistern?", + "type": "general", + "url": "https://www.currenenvironmental.com/blog/what-is-a-cistern" + }, + { + "cite": "https://www.rainbrothers.com › store › Underground-Ta...", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Need a concrete cistern ? How about a durable high-density polyethylene plastic cistern ? Check out our selection here. Precast Concrete Underground Cisterns .", + "title": "Underground Tanks", + "type": "general", + "url": "https://www.rainbrothers.com/store/Underground-Tanks-c73647298" + }, + { + "cite": "https://www.bobvila.com › articles › what-is-a-cistern", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Nov 28, 2022 — A cistern is a reservoir for collecting rainwater for household or garden use. · Fill pipe: The fill pipe (port) directs water to the cistern  ...", + "title": "Solved! What Is a Cistern?", + "type": "general", + "url": "https://www.bobvila.com/articles/what-is-a-cistern/" + }, + { + "cite": "https://www.merriam-webster.com › thesaurus › cistern", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "Synonyms for CISTERN : tub, vat, drum, cask, scuttlebutt, pipe, keg, hogshead, tun, puncheon.", + "title": "Synonyms of cistern - Merriam-Webster Thesaurus", + "type": "general", + "url": "https://www.merriam-webster.com/thesaurus/cistern" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": { + "heading": "Related searches", + "items": [ + "cistern toilet", + "cisterns brain", + "Cistern for sale", + "cistern price", + "how to pronounce cistern", + "cistern underground", + "cistern anatomy", + "what is a cistern used for" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "cistern toilet<|>cisterns brain<|>Cistern for sale<|>cistern price<|>how to pronounce cistern<|>cistern underground<|>cistern anatomy<|>what is a cistern used for", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": { + "img_urls": [ + "/search?q=cistern&sca_esv=52b056e1a97f92b3&udm=2&source=iu&ictx=1&vet=1&fir=u7VaYxkWVrmveM%252CzATw75NR6WxH1M%252C%252Fm%252F04n4fh%253BDS9gXhnQRXcw1M%252CzATw75NR6WxH1M%252C_%253B568RMSFCsqdkeM%252CYPkFp_AtnegJmM%252C_%253BMo3uC7riDHZrEM%252CUPMzGAVFxDyjkM%252C_%253BlJkV-sgFRd8NuM%252C0YmUXaW0uXBYEM%252C_%253BPt3fwumI-eHwdM%252C_tkgtlMhxMhIhM%252C_%253B_bGpjfPGwLFmSM%252CUPMzGAVFxDyjkM%252C_&usg=AI4_-kTKJ0NWxnNgfllzHAPKtVqnD9YvBg&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQ_B16BAhaEAE#vhid=u7VaYxkWVrmveM&vssid=mosaic", + "/search?q=cistern&sca_esv=52b056e1a97f92b3&udm=2&source=iu&ictx=1&vet=1&fir=u7VaYxkWVrmveM%252CzATw75NR6WxH1M%252C_%253BDS9gXhnQRXcw1M%252CzATw75NR6WxH1M%252C_%253B568RMSFCsqdkeM%252CYPkFp_AtnegJmM%252C_%253BMo3uC7riDHZrEM%252CUPMzGAVFxDyjkM%252C_%253BlJkV-sgFRd8NuM%252C0YmUXaW0uXBYEM%252C_%253BPt3fwumI-eHwdM%252C_tkgtlMhxMhIhM%252C_%253B_bGpjfPGwLFmSM%252CUPMzGAVFxDyjkM%252C_&usg=AI4_-kTMwZgJ_PzHFVkTpj7wX0epuv1Hsw&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQ_h16BAhiEAE#vhid=DS9gXhnQRXcw1M&vssid=mosaic", + "/search?q=cistern&sca_esv=52b056e1a97f92b3&udm=2&source=iu&ictx=1&vet=1&fir=u7VaYxkWVrmveM%252CzATw75NR6WxH1M%252C_%253BDS9gXhnQRXcw1M%252CzATw75NR6WxH1M%252C_%253B568RMSFCsqdkeM%252CYPkFp_AtnegJmM%252C_%253BMo3uC7riDHZrEM%252CUPMzGAVFxDyjkM%252C_%253BlJkV-sgFRd8NuM%252C0YmUXaW0uXBYEM%252C_%253BPt3fwumI-eHwdM%252C_tkgtlMhxMhIhM%252C_%253B_bGpjfPGwLFmSM%252CUPMzGAVFxDyjkM%252C_&usg=AI4_-kTMwZgJ_PzHFVkTpj7wX0epuv1Hsw&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQ_h16BAhdEAE#vhid=568RMSFCsqdkeM&vssid=mosaic", + "/search?q=cistern&sca_esv=52b056e1a97f92b3&udm=2&source=iu&ictx=1&vet=1&fir=u7VaYxkWVrmveM%252CzATw75NR6WxH1M%252C_%253BDS9gXhnQRXcw1M%252CzATw75NR6WxH1M%252C_%253B568RMSFCsqdkeM%252CYPkFp_AtnegJmM%252C_%253BMo3uC7riDHZrEM%252CUPMzGAVFxDyjkM%252C_%253BlJkV-sgFRd8NuM%252C0YmUXaW0uXBYEM%252C_%253BPt3fwumI-eHwdM%252C_tkgtlMhxMhIhM%252C_%253B_bGpjfPGwLFmSM%252CUPMzGAVFxDyjkM%252C_&usg=AI4_-kTMwZgJ_PzHFVkTpj7wX0epuv1Hsw&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQ_h16BAhgEAE#vhid=Mo3uC7riDHZrEM&vssid=mosaic", + "/search?q=cistern&sca_esv=52b056e1a97f92b3&udm=2&source=iu&ictx=1&vet=1&fir=u7VaYxkWVrmveM%252CzATw75NR6WxH1M%252C_%253BDS9gXhnQRXcw1M%252CzATw75NR6WxH1M%252C_%253B568RMSFCsqdkeM%252CYPkFp_AtnegJmM%252C_%253BMo3uC7riDHZrEM%252CUPMzGAVFxDyjkM%252C_%253BlJkV-sgFRd8NuM%252C0YmUXaW0uXBYEM%252C_%253BPt3fwumI-eHwdM%252C_tkgtlMhxMhIhM%252C_%253B_bGpjfPGwLFmSM%252CUPMzGAVFxDyjkM%252C_&usg=AI4_-kTMwZgJ_PzHFVkTpj7wX0epuv1Hsw&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQ_h16BAheEAE#vhid=lJkV-sgFRd8NuM&vssid=mosaic", + "/search?q=cistern&sca_esv=52b056e1a97f92b3&udm=2&source=iu&ictx=1&vet=1&fir=u7VaYxkWVrmveM%252CzATw75NR6WxH1M%252C_%253BDS9gXhnQRXcw1M%252CzATw75NR6WxH1M%252C_%253B568RMSFCsqdkeM%252CYPkFp_AtnegJmM%252C_%253BMo3uC7riDHZrEM%252CUPMzGAVFxDyjkM%252C_%253BlJkV-sgFRd8NuM%252C0YmUXaW0uXBYEM%252C_%253BPt3fwumI-eHwdM%252C_tkgtlMhxMhIhM%252C_%253B_bGpjfPGwLFmSM%252CUPMzGAVFxDyjkM%252C_&usg=AI4_-kTMwZgJ_PzHFVkTpj7wX0epuv1Hsw&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQ_h16BAhcEAE#vhid=Pt3fwumI-eHwdM&vssid=mosaic", + "/search?q=cistern&sca_esv=52b056e1a97f92b3&udm=2&source=iu&ictx=1&vet=1&fir=u7VaYxkWVrmveM%252CzATw75NR6WxH1M%252C_%253BDS9gXhnQRXcw1M%252CzATw75NR6WxH1M%252C_%253B568RMSFCsqdkeM%252CYPkFp_AtnegJmM%252C_%253BMo3uC7riDHZrEM%252CUPMzGAVFxDyjkM%252C_%253BlJkV-sgFRd8NuM%252C0YmUXaW0uXBYEM%252C_%253BPt3fwumI-eHwdM%252C_tkgtlMhxMhIhM%252C_%253B_bGpjfPGwLFmSM%252CUPMzGAVFxDyjkM%252C_&usg=AI4_-kTMwZgJ_PzHFVkTpj7wX0epuv1Hsw&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQ_h16BAhhEAE#vhid=_bGpjfPGwLFmSM&vssid=mosaic", + "/search?sca_esv=52b056e1a97f92b3&q=cistern&udm=2&source=univ&fir=u7VaYxkWVrmveM%252CzATw75NR6WxH1M%252C_%253BDS9gXhnQRXcw1M%252CzATw75NR6WxH1M%252C_%253B568RMSFCsqdkeM%252CYPkFp_AtnegJmM%252C_%253BMo3uC7riDHZrEM%252CUPMzGAVFxDyjkM%252C_%253BlJkV-sgFRd8NuM%252C0YmUXaW0uXBYEM%252C_%253BPt3fwumI-eHwdM%252C_tkgtlMhxMhIhM%252C_%253B_bGpjfPGwLFmSM%252CUPMzGAVFxDyjkM%252C_&usg=AI4_-kTMwZgJ_PzHFVkTpj7wX0epuv1Hsw&sa=X&ved=2ahUKEwigwZHuhPCEAxUWGVkFHaaDDWkQiR56BAhjEAI" + ], + "items": [ + "Types", + "Size", + "Pipe" + ], + "type": "panel" + }, + "error": null, + "section": "rhs", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "panel_rhs", + "text": "A cistern is a waterproof receptacle for holding liquids, usually water. Cisterns are often built to catch and store rainwater. Cisterns are distinguished from wells by their waterproof linings. Modern cisterns range in capacity from a few litres to thousands of cubic metres, effectively forming covered reservoirs.", + "title": "Cistern", + "type": "knowledge", + "url": "https://en.wikipedia.org/wiki/Cistern" + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[86cc98fe45fc].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[86cc98fe45fc].json new file mode 100644 index 00000000..8324515c --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[86cc98fe45fc].json @@ -0,0 +1,277 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": false, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 2310000000.0, + "result_estimate_time": 0.27 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "sections": [ + { + "heading": "Explanation", + "text": "Eastman was diagnosed with a progressive spinal disease in the late 1920s. In his final two years, he was in severe pain. On March 14, 1932, he shot himself in the heart." + } + ], + "sources": [ + { + "favicon": null, + "publisher": "Wikipedia", + "snippet": "In his final two years, Eastman was in intense pain caused by a disorder affecting his spine. On March 14, 1932, he shot himself i...", + "source_id": null, + "title": "George Eastman - Wikipedia", + "url": "https://en.wikipedia.org/wiki/George_Eastman#:~:text=In%20his%20final%20two%20years,designated%20a%20National%20Historic%20Landmark." + }, + { + "favicon": null, + "publisher": "The American Journal of Geriatric Psychiatry", + "snippet": "Sep 29, 2022 — Share ... On March 14, 1932 George Eastman, founder of Kodak, industrialist, entrepreneur, and fabulously wealthy phil...", + "source_id": null, + "title": "“My Work is Done. Why Wait?” Lessons Learned from Older Adults ...", + "url": "https://www.ajgponline.org/article/S1064-7481(22)00525-5/abstract#:~:text=On%20March%2014%2C%201932%20George%20Eastman%2C%20founder,and%20fabulously%20wealthy%20philanthropist%20wrote%20these%20words" + }, + { + "favicon": null, + "publisher": "George Eastman Museum", + "snippet": "In the late 1920s, Eastman was diagnosed with a progressive and irreversible spinal disease, and on March 14, 1932, he ended his o...", + "source_id": null, + "title": "About George Eastman", + "url": "https://www.eastman.org/about-george-eastman#:~:text=In%20the%20late%201920s%2C%20Eastman,Why%20wait?%E2%80%9D" + } + ], + "type": "ai_overview" + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "sectioned", + "text": "“My work is done. Why wait?” was a note left by George Eastman, the founder of Kodak, when he died by suicide in 1932.", + "title": null, + "type": "ai_overview", + "url": null + }, + { + "cite": "https://www.openculture.com › 2013/08 › the-very-con...", + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Aug 19, 2013 — On March 14th, he committed suicide by firing a single gunshot through his heart. An act as brief, and to the point, as the note he left behind.", + "title": "The Very Concise Suicide Note by Kodak Founder George ...", + "type": "general", + "url": "https://www.openculture.com/2013/08/the-very-concise-suicide-note-by-kodak-founder-george-eastman-my-work-is-done-why-wait-1932.html" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": { + "items": [ + "Why did George Eastman never marry?", + "What illness did George Eastman have?", + "What is the slogan of Kodak?", + "Who did George Eastman leave his money to?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Why did George Eastman never marry?<|>What illness did George Eastman have?<|>What is the slogan of Kodak?<|>Who did George Eastman leave his money to?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://en.wikipedia.org › wiki › George_Eastman", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "On March 14, 1932, he shot himself in the heart, leaving a note which read, \"To my friends: my work is done. Why wait ?\" Eastman is regarded as one of ...", + "title": "George Eastman", + "type": "general", + "url": "https://en.wikipedia.org/wiki/George_Eastman" + }, + { + "cite": "150+ comments · 2 years ago", + "cmpt_rank": 4, + "details": { + "items": [ + { + "text": "George Eastman's suicide note - “ To my friends, my work is ...", + "url": "https://www.reddit.com/r/pics/comments/azs473/george_eastmans_suicide_note_to_my_friends_my/" + }, + { + "text": "TIL George Eastman, the founder of Eastman Kodak ... - Reddit", + "url": "https://www.reddit.com/r/photography/comments/1v6zgr/til_george_eastman_the_founder_of_eastman_kodak/" + }, + { + "text": "TIL that George Eastman, founder of Kodak, shot himself ...", + "url": "https://www.reddit.com/r/todayilearned/comments/xv6ga/til_that_george_eastman_founder_of_kodak_shot/" + }, + { + "text": "To my friends. My work is done. Why wait? -- George Eastman", + "url": "https://www.reddit.com/r/history/comments/jqofz/to_my_friends_my_work_is_done_why_wait_george/" + }, + { + "text": "More results from www.reddit.com", + "url": "/search?q=my+work+is+done+why+wait+site:www.reddit.com&sca_esv=14324ad49b0ab5cb&sa=X&ved=2ahUKEwi2_u3S-6-LAxXosVYBHZOrB5IQrQIoBHoECFIQBQ" + } + ], + "type": "hyperlinks" + }, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": "submenu", + "text": "He was 77 when he died, and was in immense pain for the last two years. That's the important detail one should not to skim over.", + "title": "George Eastman's suicide note. To my friends my work is ...", + "type": "general", + "url": "https://www.reddit.com/r/interestingasfuck/comments/zrc5tx/george_eastmans_suicide_note_to_my_friends_my/" + }, + { + "cite": "https://www.ajgponline.org › article › abstract", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "by Y Conwell · 2022 — On March 14, 1932 George Eastman, founder of Kodak , industrialist, entrepreneur, and fabulously wealthy philanthropist wrote these words: ...", + "title": "“My Work is Done. Why Wait?” Lessons Learned from ...", + "type": "general", + "url": "https://www.ajgponline.org/article/S1064-7481(22)00525-5/abstract" + }, + { + "cite": "https://www.oddthingsiveseen.com › 2018/02 › rest-in-...", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "Feb 3, 2018 — “My work is done. Why wait?” I don't whether it's the succinctness of the message or its humility in not trying to impart any wisdom gleaned ...", + "title": "Rest In Photography: The George Eastman Suicide Note", + "type": "general", + "url": "https://www.oddthingsiveseen.com/2018/02/rest-in-photography-george-eastman.html" + }, + { + "cite": "https://www.eastman.org › about-george-eastman", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "In a note to friends, he wrote, “ My work is done. Why wait ?” During his life, George Eastman donated more than $100 million to educational and arts ...", + "title": "About George Eastman", + "type": "general", + "url": "https://www.eastman.org/about-george-eastman" + }, + { + "cite": "https://www.goodreads.com › quotes › 3184405-georg...", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "George Eastman, \"My work is done why wait ?\" He then shot himself. George Eastman. Like · likes: 2. 3 people added it.", + "title": "George Eastman, \"My work is done why wait?\" He", + "type": "general", + "url": "https://www.goodreads.com/quotes/3184405-george-eastman-my-work-is-done-why-wait-he-then" + }, + { + "cite": "https://www.victorian-cinema.net › eastman", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "A biographical guide to the earliest years of moving pictures, 1871-1901. A gateway to the world of Victorian film, behind and in front of the camera.", + "title": "George Eastman", + "type": "general", + "url": "https://www.victorian-cinema.net/eastman.php" + }, + { + "cite": "https://www.news.com.au › ... › Real-life › True-stories", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Mar 24, 2019 — The note simply read: “To my friends, my work is done. Why wait ?” Eastman's ashes were buried on the grounds of Eastman Business Park in ...", + "title": "Kodak inventor George Eastman's perfectly planned death", + "type": "general", + "url": "https://www.news.com.au/lifestyle/real-life/true-stories/the-perfectly-planned-death-of-kodak-inventor-george-eastman/news-story/8af9b9f21050cc95d4a306b21d198c08" + }, + { + "cite": "https://www.pbs.org › americanexperience › features", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "The note found by the household staff read simply: \"To my friends, My work is done--, Why wait ?\" When his casket was carried out of the Eastman House, the ...", + "title": "George Eastman | American Experience | Official Site", + "type": "general", + "url": "https://www.pbs.org/wgbh/americanexperience/features/george-eastman/" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": { + "heading": "People also search for", + "items": [ + "My work is done why wait meaning", + "My work is done why wait quotes", + "How old was George Eastman when he died", + "George Eastman Award", + "George Eastman interesting facts", + "George Eastman quotes", + "George Eastman descendants", + "When was the Kodak camera invented" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": "people_also_search_for", + "text": "My work is done why wait meaning<|>My work is done why wait quotes<|>How old was George Eastman when he died<|>George Eastman Award<|>George Eastman interesting facts<|>George Eastman quotes<|>George Eastman descendants<|>When was the Kodak camera invented", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[880beebb66bb].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[880beebb66bb].json new file mode 100644 index 00000000..c795e072 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[880beebb66bb].json @@ -0,0 +1,735 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 2690000.0, + "result_estimate_time": 0.32 + }, + "results": [ + { + "cite": "https://www.soccerbible.com › news-archive › kaka-a-h...", + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "image_strip", + "text": "Jun 9, 2009 — Kaka has worn adidas football boots since 2002, so here are his career highlights and the football boots he has worn... ... In Kaka's first season ...", + "title": "Kaka - A career in adidas football boots - 09/06/09", + "type": "general", + "url": "https://www.soccerbible.com/news-archive/2009/06/kaka-a-history-of-adidas-football-boots-09-06-09/" + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "items": [ + "What football boots does Kaka wear?", + "What boot does Neymar wear?", + "What boots does Messi wear?", + "What boots does Erling Haaland wear?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "What football boots does Kaka wear?<|>What boot does Neymar wear?<|>What boots does Messi wear?<|>What boots does Erling Haaland wear?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.footballbootsdb.com › player › Kaka", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Kaká Football Boots 2022-23: adidas Predator FT 30. Kaká is a 41-year-old Midfielder. Kaká wears adidas Predator FT 30 soccer cleats in 2022-2023.", + "title": "Kaká", + "type": "general", + "url": "https://www.footballbootsdb.com/player/Kaka/3758/" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": "small", + "text": null, + "title": "mizuno morelia", + "type": "images", + "url": null + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 1, + "sub_type": "small", + "text": null, + "title": "mizuno boots", + "type": "images", + "url": null + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 2, + "sub_type": "small", + "text": null, + "title": "adidas predator", + "type": "images", + "url": null + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 3, + "sub_type": "small", + "text": null, + "title": "copa mundial", + "type": "images", + "url": null + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 4, + "sub_type": "small", + "text": null, + "title": "ricardo kaka", + "type": "images", + "url": null + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 5, + "sub_type": "small", + "text": null, + "title": "blue", + "type": "images", + "url": null + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 6, + "sub_type": "small", + "text": null, + "title": "adidas copa", + "type": "images", + "url": null + }, + { + "cite": null, + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 7, + "sub_type": "small", + "text": null, + "title": "champions league", + "type": "images", + "url": null + }, + { + "cite": "SoccerBible", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 14, + "sub_type": "medium", + "text": null, + "title": "Kaka - A career in adidas football boots - 09/06/09 ... SoccerBible", + "type": "images", + "url": "https://www.soccerbible.com/news-archive/2009/06/kaka-a-history-of-adidas-football-boots-09-06-09/" + }, + { + "cite": "SoccerBible", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 15, + "sub_type": "medium", + "text": null, + "title": "Kaka - A career in adidas football boots - 09/06/09 ... SoccerBible", + "type": "images", + "url": "https://www.soccerbible.com/news-archive/2009/06/kaka-a-history-of-adidas-football-boots-09-06-09/" + }, + { + "cite": "Footy Headlines", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 16, + "sub_type": "medium", + "text": null, + "title": "No More Adidas? Kaka Trains in Blackout Boots - Footy Headlines Footy Headlines", + "type": "images", + "url": "https://www.footyheadlines.com/2015/11/no-more-adidas-kaka-switches-to-blackout-mizuno-boots.html" + }, + { + "cite": "YouTube", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 17, + "sub_type": "medium", + "text": null, + "title": "Kaka Boot Switch from Adidas to Mizuno? WHY? YouTube", + "type": "images", + "url": "https://www.youtube.com/watch?v=-87JMoMNxW8" + }, + { + "cite": "Dieci Auctions", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 18, + "sub_type": "medium", + "text": null, + "title": "KAKA' RICARDO 2011/2012 LIMITED EDITION #UNLEASHED MATCH WORN BOOTS *REAL MADRID C.F.* Dieci Auctions", + "type": "images", + "url": "https://10auctions.com/prodotto/kaka-ricardo-2011-2012-limited-edition-unleashed-match-worn-boots-real-madrid-c-f/" + }, + { + "cite": "Footy Headlines", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 19, + "sub_type": "medium", + "text": null, + "title": "Closer look | Kaká's Unique Adidas Copa Mundial Boots and ... Footy Headlines", + "type": "images", + "url": "https://www.footyheadlines.com/2015/08/closer-look-kakas-unique-adidas-copa-mundial-boots-shinpads.html" + }, + { + "cite": "YouTube", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": "unspecified-3", + "text": null, + "title": "KAKA goes through his ICONIC PREDATOR BOOTS ... YouTube · Pro:Direct Soccer 20 minutes, 22 seconds Dec 1, 2023", + "type": "videos", + "url": "https://www.youtube.com/watch?v=817vcbPN7cw" + }, + { + "cite": "TikTok", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 1, + "sub_type": "unspecified-3", + "text": null, + "title": "Kaka has still got it! Kaka & @Lucaneta bring the samba skills ... TikTok · prodirectsoccer 26 seconds Dec 5, 2023", + "type": "videos", + "url": "https://www.tiktok.com/@prodirectsoccer/video/7309198096108490017" + }, + { + "cite": "YouTube", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 2, + "sub_type": "unspecified-3", + "text": null, + "title": "Kaka Boot Switch from Adidas to Mizuno? WHY? YouTube · Soccer Reviews For You 8 minutes, 29 seconds Apr 10, 2017", + "type": "videos", + "url": "https://www.youtube.com/watch?v=-87JMoMNxW8" + }, + { + "cite": null, + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": "shopping", + "text": "Shop deals", + "title": "Save with deals on apparel, electronics, and more", + "type": "promo", + "url": "/search?sca_esv=52b056e1a97f92b3&q=shop+deals&sa=X&ved=2ahUKEwj93dz1hvCEAxW4EFkFHWGaAOoQ_hF6BAhSEAA" + }, + { + "cite": "10auctions.com", + "cmpt_rank": 6, + "details": { + "price": "$1,525.00", + "source": "10auctions.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Kaka' Ricardo 2011/2012 Limited Edition #Unleashed Match Worn Boots *Real Madrid C.F.*.", + "type": "products", + "url": null + }, + { + "cite": "Huckberry", + "cmpt_rank": 6, + "details": { + "price": "$240.00", + "source": "Huckberry", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Hoka Kaha 2 Gtx Hiking Boot | Duffle bag/Radiant Yellow | Size: 10 | Performance Boots | Boots.", + "type": "products", + "url": null + }, + { + "cite": "Beauty Box Korea", + "cmpt_rank": 6, + "details": { + "price": "$28.99", + "source": "Beauty Box Korea", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Kakao Friends Wool Boots.", + "type": "products", + "url": null + }, + { + "cite": "come4buy.com", + "cmpt_rank": 6, + "details": { + "price": "$55.99", + "source": "come4buy.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Boots Casual Snow na Kaka.", + "type": "products", + "url": null + }, + { + "cite": "TheShoeMart", + "cmpt_rank": 6, + "details": { + "n_reviews": "76", + "price": "$139.95", + "rating": "4.7", + "source": "TheShoeMart", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Olukai Men's Nalukai Kala Boot Sand/Bone.", + "type": "products", + "url": null + }, + { + "cite": "10auctions.com", + "cmpt_rank": 6, + "details": { + "price": "$8,050.00", + "source": "10auctions.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Kaka’ Ricardo Club World Cup 2003 Match Worn Boots * a.C. Milan *.", + "type": "products", + "url": null + }, + { + "cite": "eBay - footballmatch", + "cmpt_rank": 6, + "details": { + "price": "$149.00", + "source": "eBay - footballmatch", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Adidas Adipure Ii Ricardo Kaka Fg Elit White Boots Shoes Mens.", + "type": "products", + "url": null + }, + { + "cite": "Dressinn.com", + "cmpt_rank": 6, + "details": { + "n_reviews": "18", + "price": "$41.99", + "rating": "4.7", + "source": "Dressinn.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Man Kappa Astos Boots. SALE.", + "type": "products", + "url": null + }, + { + "cite": "https://www.ebay.com › ... › Shoes & Cleats › Men", + "cmpt_rank": 7, + "details": { + "price": "129.00", + "stock": " $20 delivery", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "Adidas AdiPure II SG Ricardo KAKA Elit Red boots mens Football/Soccers ; Item Number. 285503771170 ; Brand. adidas ; Main Color. Burgundy ; Accurate description.", + "title": "Adidas AdiPure II SG Ricardo KAKA Elit Red ...", + "type": "general", + "url": "https://www.ebay.com/itm/285503771170" + }, + { + "cite": "https://www.footballbootsdb.com › player › Kaka", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 0, + "sub_type": null, + "text": "Kaká Football Boots - Brazil Defender Kaká (42) wears adidas Predator Accuracy.1 L soccer cleats in 2022-2023.", + "title": "Kaká", + "type": "general", + "url": "https://www.footballbootsdb.com/player/Kaka/17078/" + }, + { + "cite": "https://www.ebay.com › ... › Shoes & Cleats › Men", + "cmpt_rank": 9, + "details": { + "price": "149.00", + "stock": " $20 delivery", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "Adidas AdiPure II Ricardo Kaka FG Elit White. Condition - Good - no insoles. Professional model.", + "title": "Adidas AdiPure II Ricardo Kaka FG Elit White ...", + "type": "general", + "url": "https://www.ebay.com/itm/285426753977" + }, + { + "cite": "https://bcboots.uk › products › kaka-match-worn-adida...", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 0, + "sub_type": null, + "text": "Model: Adidas Predator LZ #UNLEASH. - Player: Kaká . - Condition: Match worn & hand signed. - Season: 2011-12. - Size: UK9 / US9.5. - Soleplate: FG.", + "title": "Kaká Match Worn Adidas Predator LZ #UNLEASH ...", + "type": "general", + "url": "https://bcboots.uk/products/kaka-match-worn-adidas-predator-lz-unleash-1" + }, + { + "cite": "https://en.m.wikipedia.org › wiki › File:Kaka_boots", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 33, + "sub_rank": 0, + "sub_type": null, + "text": "File: Kaka boots .jpg ... Size of this preview: 800 × 600 pixels. Other resolutions: 320 × 240 pixels | 640 × 480 pixels | 1,024 × 768 pixels | 1,280 × 960 pixels | ...", + "title": "File:Kaka boots.jpg", + "type": "general", + "url": "https://en.m.wikipedia.org/wiki/File:Kaka_boots.jpg" + }, + { + "cite": "28.4K+ likes · 3 months ago", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 34, + "sub_rank": 0, + "sub_type": null, + "text": "28K likes, 38 comments - prodirectsoccer on December 4, 2023: \"The boots that won it all @ kaka celebrates 30 years of adidas Predator by ...", + "title": "prodirectsoccer #kaka #adidasfootball", + "type": "general", + "url": "https://www.instagram.com/prodirectsoccer/p/C0cECQZsNLN/" + }, + { + "cite": "Poshmark", + "cmpt_rank": 13, + "details": { + "price": "$30.00", + "source": "Poshmark", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 35, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Chota Kayak Boots. SALE.", + "type": "products", + "url": null + }, + { + "cite": "10auctions.com", + "cmpt_rank": 13, + "details": { + "price": "$5,630.00", + "source": "10auctions.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 36, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Kaka’ Ricardo 2004/2005 Match Worn Boots * a.C. Milan / Brazil National Team *.", + "type": "products", + "url": null + }, + { + "cite": "Huckberry", + "cmpt_rank": 13, + "details": { + "price": "$240.00", + "source": "Huckberry", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 37, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Hoka Kaha 2 Gtx Hiking Boot | Black/Black | Size: 10 | Performance Boots | Boots.", + "type": "products", + "url": null + }, + { + "cite": "eBay", + "cmpt_rank": 13, + "details": { + "price": "$79.95", + "source": "eBay", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 38, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Olukai Men's Nalukai Kapa Boot Lava Rock Black Canvas Chukka Ankle Mid Size 12.", + "type": "products", + "url": null + }, + { + "cite": "Trekkinn.com", + "cmpt_rank": 13, + "details": { + "n_reviews": "78", + "price": "$175.99", + "rating": "4.9", + "source": "Trekkinn.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 39, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Man Aku Slope Original GTX Boots Brown.", + "type": "products", + "url": null + }, + { + "cite": "eBay", + "cmpt_rank": 13, + "details": { + "n_reviews": "102", + "price": "$67.99", + "rating": "4.7", + "source": "eBay", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 40, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Olukai Men's Nalukai Kapa Boot. Low price.", + "type": "products", + "url": null + }, + { + "cite": "LN-CC", + "cmpt_rank": 13, + "details": { + "n_reviews": "640", + "price": "$230.00", + "rating": "4.3", + "source": "LN-CC", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 41, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Hoka Kaha 2 GTX Hiking Boots.", + "type": "products", + "url": null + }, + { + "cite": "HOKA", + "cmpt_rank": 13, + "details": { + "n_reviews": "640", + "price": "$240.00", + "rating": "4.3", + "source": "HOKA", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 42, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Men's Hoka Kaha 2 GTX. .", + "type": "products", + "url": null + }, + { + "cite": null, + "cmpt_rank": 14, + "details": { + "heading": "Related searches", + "items": [ + "kaka adidas predator", + "kaka mizuno", + "adidas predator boots", + "what cleats did kaka wear", + "football boots", + "jude bellingham boots" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 43, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "kaka adidas predator<|>kaka mizuno<|>adidas predator boots<|>what cleats did kaka wear<|>football boots<|>jude bellingham boots", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[8d435893480b].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[8d435893480b].json new file mode 100644 index 00000000..5800b0f1 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[8d435893480b].json @@ -0,0 +1,261 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": false, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 149000000.0, + "result_estimate_time": 0.33 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "header", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "query_suggestion", + "text": "cancer deaths per year in the us", + "title": "Did you mean:", + "type": "notice", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "sources": [ + { + "favicon": null, + "publisher": "American Cancer Society", + "snippet": "Estimated numbers of new cancer cases and deaths in 2022 (In 2022, there will be an estimated 1.9 million new cancer cases diagnos...", + "source_id": null, + "title": "Cancer Facts & Figures 2022 | American Cancer Society", + "url": "https://www.cancer.org/research/cancer-facts-statistics/all-cancer-facts-figures/cancer-facts-figures-2022.html#:~:text=Estimated%20numbers%20of%20new%20cancer,factors%2C%20early%20detection%2C%20and%20treatment" + }, + { + "favicon": null, + "publisher": "CDC", + "snippet": "Cancer * Number of deaths: 608,371. * Deaths per 100,000 population: 182.5. * Cause of death rank: 2.", + "source_id": null, + "title": "FastStats - Cancer - CDC", + "url": "https://www.cdc.gov/nchs/fastats/cancer.htm#:~:text=Cancer%20*%20Number%20of%20deaths:%20608%2C371.%20*,182.5.%20*%20Cause%20of%20death%20rank:%202." + }, + { + "favicon": null, + "publisher": "Office of Disease Prevention and Health Promotion (.gov)", + "snippet": "The cancer death rate has declined in recent decades, but over 600,000 people still die from cancer each year in the United States...", + "source_id": null, + "title": "Cancer - Healthy People 2030 | odphp.health.gov", + "url": "https://odphp.health.gov/healthypeople/objectives-and-data/browse-objectives/cancer#:~:text=The%20cancer%20death%20rate%20has%20declined%20in,cancers%20and%20in%20some%20racial/ethnic%20minority%20groups." + } + ], + "type": "ai_overview" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "flat", + "text": "According to the American Cancer Society, in 2022, there were an estimated 609,360 cancer deaths in the United States. This represents approximately 183 deaths per 100,000 people . Here are some additional statistics: Lung cancer is the leading cause of cancer death, accounting for about 25% of all cancer deaths. Prostate cancer is the second leading cause of cancer death in men, while breast cancer is the second leading cause in women. The overall cancer death rate has been declining in recent years, but it is still higher in certain racial and ethnic groups.", + "title": null, + "type": "ai_overview", + "url": null + }, + { + "cite": "https://www.cancer.org › research › cancer-facts-statistics", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Estimated numbers of new cancer cases and deaths in 2021 (In 2021, there will be an estimated 1.9 million new cancer cases diagnosed and 608,570 cancer deaths  ...", + "title": "Cancer Facts & Figures 2021", + "type": "general", + "url": "https://www.cancer.org/research/cancer-facts-statistics/all-cancer-facts-figures/cancer-facts-figures-2021.html" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": { + "items": [ + "How many cancer deaths in the US per year?", + "Is cancer the #1 leading cause of death among Americans?", + "What are the top 5 cancer deaths in the US?", + "What US state has the highest cancer rate?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "How many cancer deaths in the US per year?<|>Is cancer the #1 leading cause of death among Americans?<|>What are the top 5 cancer deaths in the US?<|>What US state has the highest cancer rate?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.cdc.gov › nchs › hus › topics › cancer-deat...", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Age-adjusted cancer deaths per 100,000 population decreased from 173.5 in 2009 to 155.8 in 2016, and then decreased at a faster rate to 146.2 in 2019. A total ...", + "title": "Cancer deaths - Health, United States", + "type": "general", + "url": "https://www.cdc.gov/nchs/hus/topics/cancer-deaths.htm" + }, + { + "cite": "https://seer.cancer.gov › statfacts › html › all", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "The death rate was 146.0 per 100,000 men and women per year . These rates are age-adjusted and based on 2017–2021 cases and 2018–2022 deaths. Lifetime Risk of ...", + "title": "Cancer Stat Facts: Cancer of Any Site", + "type": "general", + "url": "https://seer.cancer.gov/statfacts/html/all.html" + }, + { + "cite": "https://www.cdc.gov › nchs › fastats › cancer", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "Cancer · Number of deaths: 608,371 · Deaths per 100,000 population: 182.5 · Cause of death rank: 2.", + "title": "FastStats - Cancer", + "type": "general", + "url": "https://www.cdc.gov/nchs/fastats/cancer.htm" + }, + { + "cite": "https://www.cancer.org › research › cancer-facts-statistics", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "Estimated numbers of new cancer cases and deaths in 2022 (In 2022, there will be an estimated 1.9 million new cancer cases diagnosed and 609,360 cancer deaths  ...", + "title": "Cancer Facts & Figures 2022", + "type": "general", + "url": "https://www.cancer.org/research/cancer-facts-statistics/all-cancer-facts-figures/cancer-facts-figures-2022.html" + }, + { + "cite": "https://usafacts.org › Health › Articles", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "While total cancer deaths grew by 10.6% between 2000 and 2021, the annual mortality rate fell from 198.8 to 144.2 per 100,000 people — a 27.5% decline. The ...", + "title": "how have cancer rates and mortality changed over time?", + "type": "general", + "url": "https://usafacts.org/articles/how-have-cancer-rates-changed-over-time/" + }, + { + "cite": "https://seer.cancer.gov › statfacts › html › common", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "In 2024, an estimated 611,720 people will die of cancer in the United States . Lung and bronchus cancer is responsible for the most deaths with 125,070 people ...", + "title": "Cancer Stat Facts: Common Cancer Sites", + "type": "general", + "url": "https://seer.cancer.gov/statfacts/html/common.html" + }, + { + "cite": "https://www.webmd.com › Cancer › Reference", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Jun 19, 2024 — That's 1 out of every 6 deaths. Over 610,000 cancer deaths happen in the U.S. each year and about 80,000 in Canada. The rest happens in ...", + "title": "How Many People Die of Cancer a Year?", + "type": "general", + "url": "https://www.webmd.com/cancer/how-many-cancer-deaths-per-year" + }, + { + "cite": "https://acsjournals.onlinelibrary.wiley.com › doi › full", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "by RL Siegel · 2024 · Cited by 3811 — In 2024, 2,001,140 new cancer cases and 611,720 cancer deaths are projected to occur in the United States . Cancer mortality continued to decline ...", + "title": "Cancer statistics, 2024 - CA: A Cancer Journal for Clinicians", + "type": "general", + "url": "https://acsjournals.onlinelibrary.wiley.com/doi/full/10.3322/caac.21820" + }, + { + "cite": "https://pubmed.ncbi.nlm.nih.gov › ...", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "by RL Siegel · 2023 · Cited by 12713 — In 2023, 1,958,310 new cancer cases and 609,820 cancer deaths are projected to occur in the United States . Cancer incidence increased for prostate cancer by 3% ...", + "title": "Cancer statistics, 2023", + "type": "general", + "url": "https://pubmed.ncbi.nlm.nih.gov/36633525/" + }, + { + "cite": null, + "cmpt_rank": 13, + "details": { + "heading": "People also search for", + "items": [ + "Cancer deaths per year worldwide", + "1 in how many get cancer", + "This year in the us how many deaths due to cancer every minute", + "Cancer deaths US 2023", + "Cancer statistics worldwide", + "How many people are diagnosed with cancer each year", + "Cancer rates over the last 100 years graph", + "Cancer statistics 2024" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "people_also_search_for", + "text": "Cancer deaths per year worldwide<|>1 in how many get cancer<|>This year in the us how many deaths due to cancer every minute<|>Cancer deaths US 2023<|>Cancer statistics worldwide<|>How many people are diagnosed with cancer each year<|>Cancer rates over the last 100 years graph<|>Cancer statistics 2024", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[97404b7b7c61].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[97404b7b7c61].json deleted file mode 100644 index a68e9ccb..00000000 --- a/tests/__snapshots__/test_parse_serp/test_parse_serp[97404b7b7c61].json +++ /dev/null @@ -1,485 +0,0 @@ -{ - "features": { - "captcha": false, - "infinity_scroll": false, - "language": "en", - "main_layout": "standard", - "notice_no_results": false, - "notice_server_error": false, - "notice_shortened_query": false, - "overlay_precise_location": false, - "result_estimate_count": 2560000000.0, - "result_estimate_time": 0.38 - }, - "results": [ - { - "cite": null, - "cmpt_rank": 0, - "details": { - "citations": [ - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "1", - "2", - "15" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "5", - "14", - "9" - ] - } - ], - "sources": [ - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.weather.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Weather Service (.gov)", - "snippet": "While all colors are scattered by air molecules, violet and blue are scattered most. The sky looks blue, not violet, because our e...", - "source_id": "2", - "title": "Why Is The Sky Blue? - National Weather Service", - "url": "https://www.weather.gov/fgz/SkyBlue" - }, - { - "favicon": "https://encrypted-tbn3.gstatic.com/faviconV2?url=https://spaceplace.nasa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "NASA Space Place (.gov)", - "snippet": "Sunlight reaches Earth's atmosphere and is scattered in all directions by all the gases and particles in the air. Blue light is sc...", - "source_id": "5", - "title": "Why Is the Sky Blue? | NASA Space Place – NASA Science for Kids", - "url": "https://spaceplace.nasa.gov/blue-sky/" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.nesdis.noaa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Environmental Satellite, Data, and Information Service (.gov)", - "snippet": "The sky appears blue because of the scattering of sunlight by gases and particles in Earth's atmosphere. Blue light is scattered m...", - "source_id": "1", - "title": "Why Is the Sky Blue? | NESDIS | National Environmental Satellite, Data, and Information Service", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue#:~:text=The%20sky%20appears%20blue%20because%20of%20the,we%20can%20see%20look%20blue%20or%20violet**" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.montrealsciencecentre.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Montreal Science Centre", - "snippet": "Wrapped around the Earth, the atmosphere is made up of a thin layer of gases, mainly nitrogen and oxygen. This is where the air we...", - "source_id": "9", - "title": "Why is the Sky Blue? - Montreal Science Centre", - "url": "https://www.montrealsciencecentre.com/blog/why-the-sky-blue" - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.facebook.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Facebook", - "snippet": "The sky appears blue because of a phenomenon called Rayleigh scattering, which is the scattering of light by small particles or mo...", - "source_id": "15", - "title": "Why is the sky blue? It's an age old question that actually has a very ...", - "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.rmg.co.uk&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Royal Museums Greenwich", - "snippet": "This content is hosted by a third party. Please allow all cookies to watch the video. ... These different colours have different w...", - "source_id": "14", - "title": "Why is the sky blue? | Royal Observatory", - "url": "https://www.rmg.co.uk/stories/space-astronomy/why-sky-blue" - } - ], - "type": "ai_overview" - }, - "error": null, - "section": "main", - "serp_rank": 0, - "sub_rank": 0, - "sub_type": "flat", - "text": "The sky is blue due to Rayleigh scattering , where Earth's atmosphere scatters sunlight, with shorter blue and violet wavelengths scattering more effectively than longer red wavelengths, filling the sky with blue light, though our eyes see blue more than violet. Here's a step-by-step breakdown: Sunlight is White Light: Sunlight appears white but contains all the colors of the rainbow (red, orange, yellow, green, blue, indigo, violet). Light Travels in Waves: Each color travels as a wave, with blue/violet having shorter, smaller waves and red/orange having longer waves. Atmosphere Scattering: As sunlight enters the atmosphere, it hits tiny molecules of nitrogen and oxygen . Blue Light Scattered Most: These small gas molecules scatter the shorter blue and violet wavelengths in all directions much more than they scatter the longer red and yellow wavelengths. Why Blue, Not Violet: While violet is scattered even more, our eyes are more sensitive to blue light, and the sun emits slightly more blue light, making the sky appear blue. Pale Near Horizon: Near the horizon, the sky looks paler because the blue light has traveled through more atmosphere, scattering away, leaving more white and yellow light to reach your eyes. This process explains why sunsets are red (blue is scattered away, leaving reds and oranges) and why space is black (no atmosphere to scatter light).", - "title": null, - "type": "ai_overview", - "url": null - }, - { - "cite": null, - "cmpt_rank": 1, - "details": { - "items": [ - "Why is the sky blue short answer?", - "How to explain to a kid why the sky is blue?", - "Why the sky looks bluer in fall and winter - WSAV-TV WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo... WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo...", - "Why is the sky Blue? desy.de https://www.desy.de › user › projects › Physics › General desy.de https://www.desy.de › user › projects › Physics › General" - ], - "type": "text" - }, - "error": null, - "section": "main", - "serp_rank": 1, - "sub_rank": 0, - "sub_type": null, - "text": "Why is the sky blue short answer?<|>How to explain to a kid why the sky is blue?<|>Why the sky looks bluer in fall and winter - WSAV-TV WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo... WSAV-TV https://www.wsav.com › weather-news › why-the-sky-lo...<|>Why is the sky Blue? desy.de https://www.desy.de › user › projects › Physics › General desy.de https://www.desy.de › user › projects › Physics › General", - "title": null, - "type": "people_also_ask", - "url": null - }, - { - "cite": "https://spaceplace.nasa.gov › blue-sky", - "cmpt_rank": 2, - "details": null, - "error": null, - "section": "main", - "serp_rank": 2, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://spaceplace.nasa.gov/blue-sky/" - }, - { - "cite": "https://math.ucr.edu › home › baez › physics › General", - "cmpt_rank": 3, - "details": null, - "error": null, - "section": "main", - "serp_rank": 3, - "sub_rank": 0, - "sub_type": null, - "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" - }, - { - "cite": "160+ comments · 2 years ago", - "cmpt_rank": 4, - "details": { - "items": [ - { - "text": "Eli5: why is the sky blue? : r/explainlikeimfive - Reddit", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1iytxes/eli5_why_is_the_sky_blue/" - }, - { - "text": "I know why the sky is blue but why is the light that ...", - "url": "https://www.reddit.com/r/askscience/comments/1vcwur/i_know_why_the_sky_is_blue_but_why_is_the_light/" - }, - { - "text": "More results from www.reddit.com", - "url": "/search?q=why+is+the+sky+blue?+site:www.reddit.com&sca_esv=540cd09fb70162eb&sa=X&ved=2ahUKEwjSkYDkv5mSAxWXSDABHUvgByYQrQIoAnoECEMQAw" - } - ], - "type": "hyperlinks" - }, - "error": null, - "section": "main", - "serp_rank": 4, - "sub_rank": 0, - "sub_type": "submenu", - "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", - "title": "Why is the sky blue? Do I understand it correctly", - "type": "general", - "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" - }, - { - "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", - "cmpt_rank": 5, - "details": null, - "error": null, - "section": "main", - "serp_rank": 5, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time.", - "title": "Why Is the Sky Blue? | NESDIS", - "type": "general", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 6, - "sub_rank": 0, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Sky Blue? Credit to Discover.", - "type": "perspectives", - "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 7, - "sub_rank": 1, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why the Sky Looks Blue\n\n#atmosphere #sky #bluesky #whyskyisblue #space", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DTsZrgoD0xa/" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 8, - "sub_rank": 2, - "sub_type": "what_people_are_saying", - "text": null, - "title": "The REAL Reason the Sky Looks Blue", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 9, - "sub_rank": 3, - "sub_type": "what_people_are_saying", - "text": null, - "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", - "type": "perspectives", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 10, - "sub_rank": 4, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", - "type": "perspectives", - "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 11, - "sub_rank": 5, - "sub_type": "what_people_are_saying", - "text": null, - "title": "The #sky achieves its color from sunlight scattering off Earth's atmosphere , a process called #Rayleigh scattering , where tiny air molecules would scatter shorter blue #wavelengths more than longer red ones, making the sky blue during the day . \n\nWe do know so .", - "type": "perspectives", - "url": "https://x.com/PARODY_CAIT/status/2005596656985678030?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 12, - "sub_rank": 6, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 13, - "sub_rank": 7, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why are #sunsets red but the #sky is blue? They are both caused by the same thing. \n\nThe light from the sun is white-ish. That means it's a combination of red, green and blue light. It turns out that the blue light is scattered by the air (nitrogen and oxygen molecules). This scattered blue light enters your eyes and you see the sky as blue.\n\nFor the sunset, the sun is low in the sky and passes through MORE air and that means that MORE of the blue light has been scattered. With less blue light from the sun, it looks more red.\n\nHere's a great physics demo. Put some powdered coffee creamer in water and shine a flashlight through it. If you look at the water from the side, it looks blueish. Looking into the flashlight it looks more red. Just like the sunset and sky.\n\n#physics #science #stem #physicsdemo", - "type": "perspectives", - "url": "https://www.instagram.com/p/DR2aqixgJUa/" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 14, - "sub_rank": 8, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/o1pTCw8cg10" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 15, - "sub_rank": 9, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Look at the sky. It appears blue during the daytime and reddish during sunrise and sunset. Why? Let me introduce you today to a British man who explained these phenomena very clearly. Further, his work for classical black-body radiation, later played an imp", - "type": "perspectives", - "url": "https://www.facebook.com/100064519939797/posts/look-at-the-sky-it-appears-blue-during-the-daytime-and-reddish-during-sunrise-an/1257662263061093/" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 16, - "sub_rank": 10, - "sub_type": "what_people_are_saying", - "text": null, - "title": "The sky looks blue because of Rayleigh scattering. Sunlight is actually white, not yellow. Clouds weigh millions of kilograms. The sky on Mars looks reddish-pink. The highest clouds are called noctilucent clouds. Rainbows are full circles, not arcs. You can nev", - "type": "perspectives", - "url": "https://www.facebook.com/teacherceppee/posts/the-sky-looks-blue-because-of-rayleigh-scatteringsunlight-is-actually-white-not-/1429848428511661/" - }, - { - "cite": null, - "cmpt_rank": 6, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 17, - "sub_rank": 11, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why the sky is blue", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DSOGSiPkTjw/?hl=en" - }, - { - "cite": "https://kids.nationalgeographic.com › books › article", - "cmpt_rank": 7, - "details": null, - "error": null, - "section": "main", - "serp_rank": 18, - "sub_rank": 0, - "sub_type": null, - "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today.", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://kids.nationalgeographic.com/books/article/sky" - }, - { - "cite": "859.4K+ views · 5 years ago", - "cmpt_rank": 8, - "details": null, - "error": null, - "section": "main", - "serp_rank": 19, - "sub_rank": 0, - "sub_type": "video", - "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" - }, - { - "cite": "https://www.weather.gov › fgz › SkyBlue", - "cmpt_rank": 9, - "details": null, - "error": null, - "section": "main", - "serp_rank": 20, - "sub_rank": 0, - "sub_type": null, - "text": "The sky looks blue, not violet, because our eyes are more sensitive to blue light (and the sun also emits more energy as blue light than as violet). Read more", - "title": "Why Is The Sky Blue?", - "type": "general", - "url": "https://www.weather.gov/fgz/SkyBlue" - }, - { - "cite": "50+ reactions · 1 year ago", - "cmpt_rank": 10, - "details": null, - "error": null, - "section": "main", - "serp_rank": 21, - "sub_rank": 0, - "sub_type": "video", - "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", - "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", - "type": "general", - "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" - }, - { - "cite": "https://www.uu.edu › dept › physics › scienceguys", - "cmpt_rank": 11, - "details": null, - "error": null, - "section": "main", - "serp_rank": 22, - "sub_rank": 0, - "sub_type": null, - "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", - "title": "Why is the sky blue on Earth, but black in space or ...", - "type": "general", - "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" - }, - { - "cite": null, - "cmpt_rank": 12, - "details": { - "heading": "People also search for", - "items": [ - "Why is the sky blue ai", - "Why is the sky blue Class 10", - "Why is the sky blue for kids", - "Why is the sky blue reflection of ocean", - "Why is the sky blue and not violet", - "Why is the sky blue Army", - "Why is the sky blue long answer", - "Why is the sky blue essay" - ], - "type": "text" - }, - "error": null, - "section": "footer", - "serp_rank": 23, - "sub_rank": 0, - "sub_type": "people_also_search_for", - "text": "Why is the sky blue ai<|>Why is the sky blue Class 10<|>Why is the sky blue for kids<|>Why is the sky blue reflection of ocean<|>Why is the sky blue and not violet<|>Why is the sky blue Army<|>Why is the sky blue long answer<|>Why is the sky blue essay", - "title": null, - "type": "searches_related", - "url": null - } - ] -} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[984065877aad].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[984065877aad].json deleted file mode 100644 index 4fdf4e37..00000000 --- a/tests/__snapshots__/test_parse_serp/test_parse_serp[984065877aad].json +++ /dev/null @@ -1,624 +0,0 @@ -{ - "features": { - "captcha": false, - "infinity_scroll": false, - "language": "en", - "main_layout": "standard", - "notice_no_results": false, - "notice_server_error": false, - "notice_shortened_query": false, - "overlay_precise_location": false, - "result_estimate_count": 2000000000.0, - "result_estimate_time": 0.27 - }, - "results": [ - { - "cite": null, - "cmpt_rank": 0, - "details": { - "citations": [ - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "2", - "13", - "5", - "6", - "4" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "3", - "14", - "9", - "10" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "15", - "11", - "46", - "47", - "48" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "53", - "54", - "55" - ] - } - ], - "sources": [ - { - "favicon": "https://encrypted-tbn3.gstatic.com/faviconV2?url=https://www.franklin.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Franklin University", - "snippet": "One of the most significant is access to more job opportunities and lower unemployment rates. According to the Bureau of Labor Sta...", - "source_id": "5", - "title": "Is College Worth The Cost - Franklin University", - "url": "https://www.franklin.edu/blog/is-college-worth-the-cost" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://bigfuture.collegeboard.org&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "College Board", - "snippet": "College does matter and is absolutely worth it - if you choose a program that matches your career goals, graduate on time, and avo...", - "source_id": "4", - "title": "Why College Is Important - BigFuture", - "url": "https://bigfuture.collegeboard.org/plan-for-college/get-started/why-college-is-important#:~:text=College%20does%20matter%20and%20is%20absolutely%20worth,requires%20education%20or%20training%20beyond%20high%20school." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.reddit.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Reddit", - "snippet": "College broadly speaking is of questionable value in many cases but accounting is one of the few majors where the traditional dyna...", - "source_id": "15", - "title": "Is college even worth it anymore? : r/findapath - Reddit", - "url": "https://www.reddit.com/r/findapath/comments/17foihd/is_college_even_worth_it_anymore/" - }, - { - "favicon": "https://encrypted-tbn3.gstatic.com/faviconV2?url=https://www.ppic.org&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Public Policy Institute of California", - "snippet": "College is a good investment. Over the last decade, the college wage premium—the difference in wages between college graduates and...", - "source_id": "3", - "title": "Is College Worth It? - Public Policy Institute of California", - "url": "https://www.ppic.org/publication/is-college-worth-it/" - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.knox.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Knox College", - "snippet": "The sentiment that the expense of attaining a four-year degree, particularly at a private school, isn't worth the outcome is certa...", - "source_id": "2", - "title": "Yes, College is Worth It: Busting Myths about Higher Education - Features", - "url": "https://www.knox.edu/magazine/spring-2018/features/yes-college-is-worth-it" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.nbcnews.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "NBC News", - "snippet": "Poll: In a dramatic shift, Americans no longer see four-year college degrees as worth the cost. The latest NBC News poll shows two...", - "source_id": "9", - "title": "Poll: In a dramatic shift, Americans no longer see four-year college ...", - "url": "https://www.nbcnews.com/politics/politics-news/poll-dramatic-shift-americans-no-longer-see-four-year-college-degrees-rcna243672" - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.youtube.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "YouTube", - "snippet": "Trades, apprenticeships, and businesses such as HVAC, plumbing, landscaping, and car washes are often overlooked career paths. The...", - "source_id": "14", - "title": "Is college still worth it, or is it just an expensive tradition we never ...", - "url": "https://www.youtube.com/shorts/Ep0750U7M18#:~:text=Trades%2C%20apprenticeships%2C%20and%20businesses%20such%20as%20HVAC%2C,to%20which%20the%20MBA%20graduate%20is%20applying." - }, - { - "favicon": "https://encrypted-tbn3.gstatic.com/faviconV2?url=https://magazine.utah.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "The University of Utah", - "snippet": "Beyond the financial gains, her research uncovers another compelling reality: degree holders experience much higher measures of pe...", - "source_id": "6", - "title": "Is College Still Worth It? - The University of Utah Magazine", - "url": "https://magazine.utah.edu/issues/fall-2024/is-college-still-worth-it/#:~:text=Beyond%20the%20financial%20gains%2C%20her%20research%20uncovers,their%20futures%20compared%20to%20those%20without%20degrees." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.youtube.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "YouTube", - "snippet": "without a name that no one has ever heard of It can be private It can be public It can be religious It can be secular It can be an...", - "source_id": "13", - "title": "Is College Still Worth It? Part 1", - "url": "https://www.youtube.com/watch?v=AOVD1NMqYDk" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.usatoday.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "USA Today", - "snippet": "A political divide. People across age, race and gender demographics were all less likely to see college as important and worthwhil...", - "source_id": "10", - "title": "Is college worthwhile? Two-thirds of Americans say no, new poll finds", - "url": "https://www.usatoday.com/story/news/nation/2025/12/02/college-degree-worth-cost-poll/87567664007/" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.chicagofed.org&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Chicago - Federal Reserve Bank", - "snippet": "For example, research from College Board indicates that tuition sticker prices were rising from 2006-07 through 2020-2021 (and fel...", - "source_id": "11", - "title": "Is College a Worthwhile Investment? Examining the Benefits and ...", - "url": "https://www.chicagofed.org/publications/chicago-fed-insights/2024/policy-brief-is-college-a-worthwhile-investment#:~:text=For%20example%2C%20research%20from%20College%20Board%20indicates,for%20both%20public%20and%20private%20non%2Dprofit%20colleges." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.forbes.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Forbes", - "snippet": "The analysis demonstrates that certain majors are likelier to pay off. Among four-year degrees, most engineering, computer science...", - "source_id": "46", - "title": "Is College Worth It? That’s The Wrong Question", - "url": "https://www.forbes.com/sites/prestoncooper2/2024/05/17/is-college-worth-it-thats-the-wrong-question/#:~:text=The%20analysis%20demonstrates%20that%20certain%20majors%20are,their%20net%20lifetime%20earnings%20by%20over%20$500%2C000." - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.wcnc.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "WCNC", - "snippet": "Researchers say investing in a college degree still pays off well with better returns that the stock market. Their second takeaway...", - "source_id": "47", - "title": "Why a 4-year college degree is still worth the money", - "url": "https://www.wcnc.com/article/money/4-year-college-degree-worth-it-financial-analysis-money-education/275-8e064c56-b11b-4dd7-b30c-d9257dfe87ae#:~:text=Researchers%20say%20investing%20in%20a%20college%20degree,bringing%20home%20the%20most%20money%20on%20average." - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.collegeflightpath.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "College Flight Path", - "snippet": "High-Paying College Majors & ROI by Major Certain fields have consistently shown high financial returns. Data from leading researc...", - "source_id": "48", - "title": "College ROI | Understanding Return on Investment for College Degrees", - "url": "https://www.collegeflightpath.com/cfp-blog/understanding-the-roi-of-college-majors-financial-considerations#:~:text=High%2DPaying%20College%20Majors%20&%20ROI%20by%20Major,at%20the%20top%20of%20the%20ROI%20charts." - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.insidehighered.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Inside Higher Ed", - "snippet": "Who's paying for it? When confronted with this high price, many students say they turn to loans to pay for their degree, which can...", - "source_id": "53", - "title": "Cost of higher education not worth it to students", - "url": "https://www.insidehighered.com/news/student-success/college-experience/2024/05/29/cost-higher-education-not-worth-it-students#:~:text=Who's%20paying%20for%20it?%20When%20confronted%20with,graduating%2C%20hurting%20the%20overall%20return%20on%20investment." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.ivywise.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "IvyWise", - "snippet": "Studies have shown that a lack of clarity around one's academic interests and career goals can delay graduation, leading to increa...", - "source_id": "54", - "title": "Academic Advising for College Students", - "url": "https://www.ivywise.com/admissions-counseling/academic-advising/#:~:text=Studies%20have%20shown%20that%20a%20lack%20of,year%20of%20college%20can%20cost%20around%20$68%2C000." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.reddit.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Reddit", - "snippet": "If you go into with no career plan, no idea of how your degree is going to help you, and a terribly cynical attitude about it, the...", - "source_id": "55", - "title": "Is college a scam? : r/careeradvice", - "url": "https://www.reddit.com/r/careeradvice/comments/1o1nm6d/is_college_a_scam/#:~:text=If%20you%20go%20into%20with%20no%20career,expect%20it%20to%20be%20a%20golden%20ticket." - } - ], - "type": "ai_overview" - }, - "error": null, - "section": "main", - "serp_rank": 0, - "sub_rank": 0, - "sub_type": "flat", - "text": "A college degree is generally worth the investment , offering a ~73% higher average salary, better job security, and over $1 million more in lifetime earnings compared to a high school diploma. While costs are high, graduates also report better health, increased networking, and better career mobility. However, the value depends on choosing a high-ROI major, graduating on time, and managing debt. Key Considerations Financial Return: In 2023, bachelor's degree holders earned 61% more on average than those with only a high school diploma. Career Security: The unemployment rate for those with a bachelor’s degree (2.2%) is nearly half that of those with only a high school diploma (4%) . Alternatives: Trades and specialized training can offer high incomes ($60,000–$70,000+) with little to no debt, providing a faster route to financial stability in some cases. Declining Sentiment: Despite the statistical benefits, about two-thirds of Americans in a 2025 poll questioned the value of a four-year degree due to rising costs and political factors. When is it Most Worth It? High-ROI Majors: Fields like engineering, computer science, and nursing often provide a high return. Lower-Cost Options: Starting at a community college or attending public, in-state institutions significantly reduces debt. When is it Less Worth It? High Debt/Low Salary: Taking on excessive student loans for a degree in a field with low earning potential can lead to a negative return on investment. Lack of Clear Plan: Attending without a clear career goal often leads to longer graduation times and wasted tuition.", - "title": null, - "type": "ai_overview", - "url": null - }, - { - "cite": "https://www.ppic.org › Publication", - "cmpt_rank": 1, - "details": null, - "error": null, - "section": "main", - "serp_rank": 1, - "sub_rank": 0, - "sub_type": null, - "text": "Apr 21, 2025 — A college degree does not guarantee financial security , but for the vast majority of students it represents their best chance of achieving economic prosperity. Read more", - "title": "Is College Worth It?", - "type": "general", - "url": "https://www.ppic.org/publication/is-college-worth-it/" - }, - { - "cite": null, - "cmpt_rank": 2, - "details": { - "items": [ - "Why isn't Gen Z going to college?", - "Is $70,000 a good salary out of college?", - "Can you make $100,000 a year without a degree?", - "Why College Is Important - BigFuture BigFuture - College Board https://bigfuture.collegeboard.org › get-started › why-col... BigFuture - College Board https://bigfuture.collegeboard.org › get-started › why-col..." - ], - "type": "text" - }, - "error": null, - "section": "main", - "serp_rank": 2, - "sub_rank": 0, - "sub_type": null, - "text": "Why isn't Gen Z going to college?<|>Is $70,000 a good salary out of college?<|>Can you make $100,000 a year without a degree?<|>Why College Is Important - BigFuture BigFuture - College Board https://bigfuture.collegeboard.org › get-started › why-col... BigFuture - College Board https://bigfuture.collegeboard.org › get-started › why-col...", - "title": null, - "type": "people_also_ask", - "url": null - }, - { - "cite": "Reddit  ·  r/findapath  ·  180+ comments  ·  9 months ago", - "cmpt_rank": 3, - "details": null, - "error": null, - "section": "main", - "serp_rank": 3, - "sub_rank": 0, - "sub_type": null, - "text": null, - "title": "Is college, especially in the U.S., even worth it anymore?", - "type": "discussions_and_forums", - "url": "https://www.reddit.com/r/findapath/comments/1k5lw18/is_college_especially_in_the_us_even_worth_it/" - }, - { - "cite": "Quora  ·  5 answers  ·  1 year ago", - "cmpt_rank": 3, - "details": null, - "error": null, - "section": "main", - "serp_rank": 4, - "sub_rank": 1, - "sub_type": null, - "text": null, - "title": "Is a college degree still worth the cost and effort, given the decreasing ...", - "type": "discussions_and_forums", - "url": "https://www.quora.com/Is-a-college-degree-still-worth-the-cost-and-effort-given-the-decreasing-value-and-rising-costs" - }, - { - "cite": "Quora  ·  1 answer  ·  8 months ago", - "cmpt_rank": 3, - "details": null, - "error": null, - "section": "main", - "serp_rank": 5, - "sub_rank": 2, - "sub_type": null, - "text": null, - "title": "Is a college degree really worth it in 2025?", - "type": "discussions_and_forums", - "url": "https://www.quora.com/Is-a-college-degree-really-worth-it-in-2025" - }, - { - "cite": "490+ comments · 2 years ago", - "cmpt_rank": 4, - "details": { - "items": [ - { - "text": "Is college, especially in the U.S., even worth it ...", - "url": "https://www.reddit.com/r/findapath/comments/1k5lw18/is_college_especially_in_the_us_even_worth_it/" - }, - { - "text": "Are university degrees still worth it in 2025? : r ...", - "url": "https://www.reddit.com/r/education/comments/1khliaq/are_university_degrees_still_worth_it_in_2025/" - }, - { - "text": "More results from www.reddit.com", - "url": "/search?q=is+college+worth+it+site:www.reddit.com&sca_esv=a23961d46812ec7b&sa=X&ved=2ahUKEwjWxeTJisaSAxV6d_UHHYb1DF0QrQIoAnoECEUQAw" - } - ], - "type": "hyperlinks" - }, - "error": null, - "section": "main", - "serp_rank": 6, - "sub_rank": 0, - "sub_type": "submenu", - "text": "College is worth it but getting a degree definitely does not guarantee a good job. You have to get the right degree. Only certain degrees are ... Read more", - "title": "Is college even worth it anymore? : r/findapath", - "type": "general", - "url": "https://www.reddit.com/r/findapath/comments/17foihd/is_college_even_worth_it_anymore/" - }, - { - "cite": "https://www.pewresearch.org › ... › Higher Education", - "cmpt_rank": 5, - "details": null, - "error": null, - "section": "main", - "serp_rank": 7, - "sub_rank": 0, - "sub_type": null, - "text": "May 23, 2024 — Americans have mixed views on the importance of having a degree . 47% say the cost is worth it only if someone doesn't have to take out ...", - "title": "Is a College Degree Worth It in 2024?", - "type": "general", - "url": "https://www.pewresearch.org/social-trends/2024/05/23/is-college-worth-it-2/" - }, - { - "cite": "https://libertystreeteconomics.newyorkfed.org › 2025/04", - "cmpt_rank": 6, - "details": null, - "error": null, - "section": "main", - "serp_rank": 8, - "sub_rank": 0, - "sub_type": null, - "text": "Apr 16, 2025 — A look at whether college is still worth it using an analysis of the costs, benefits, and return for the typical college student.", - "title": "Is College Still Worth It? - Liberty Street Economics", - "type": "general", - "url": "https://libertystreeteconomics.newyorkfed.org/2025/04/is-college-still-worth-it/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 9, - "sub_rank": 0, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Opinion | Is a Four-Year Degree Worth It?", - "type": "perspectives", - "url": "https://www.wsj.com/opinion/is-a-four-year-degree-worth-it-6af09e3b" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 10, - "sub_rank": 1, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Is College Really Worth the Cost?", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/uH6yxZwSMyc" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 11, - "sub_rank": 2, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Opinion | Is college worth it? This quiz shows the reality.", - "type": "perspectives", - "url": "https://www.washingtonpost.com/opinions/interactive/2026/gapminder-college-education-cost-worthwhile/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 12, - "sub_rank": 3, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Is A College Degree Worth It? (Hint: yes, but it depends)", - "type": "perspectives", - "url": "https://www.linkedin.com/pulse/college-degree-worth-hint-yes-depends-doan-winkel-cw2bc" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 13, - "sub_rank": 4, - "sub_type": "what_people_are_saying", - "text": null, - "title": "What jobs are actually worth going to college for now?", - "type": "perspectives", - "url": "https://www.reddit.com/r/Fire/comments/1qpyjqv/what_jobs_are_actually_worth_going_to_college_for/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 14, - "sub_rank": 5, - "sub_type": "what_people_are_saying", - "text": null, - "title": "From @WSJopinion: Is a college degree worth it? In this week’s Future View column, students discuss the value of higher education at a time when universities are expensive and federal student loan debt is skyrocketing", - "type": "perspectives", - "url": "https://x.com/WSJ/status/2009012670037430446?ref_src=twsrc%5Egoogle%7Ctwcamp%5Eserp%7Ctwgr%5Etweet" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 15, - "sub_rank": 6, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Is college still worth it? Debt-adjusted earnings say ‘yes’", - "type": "perspectives", - "url": "https://www.luminafoundation.org/news-and-views/is-college-still-worth-it-debt-adjusted-earnings-say-yes/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 16, - "sub_rank": 7, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Is A College Degree Worth It In This Day and Age? | by Adeen Kennedy | Dec, 2025", - "type": "perspectives", - "url": "https://medium.com/@adeenmckenziekennedy/is-a-college-degree-worth-it-in-this-day-and-age-f3c01cd6f928" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 17, - "sub_rank": 8, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Americans don't think college is worth it. It is?", - "type": "perspectives", - "url": "https://www.reddit.com/r/Salary/comments/1pbmb9p/americans_dont_think_college_is_worth_it_it_is/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 18, - "sub_rank": 9, - "sub_type": "what_people_are_saying", - "text": null, - "title": "College really worth it?", - "type": "perspectives", - "url": "https://www.reddit.com/r/flying/comments/1qwzesb/college_really_worth_it/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 19, - "sub_rank": 10, - "sub_type": "what_people_are_saying", - "text": null, - "title": "The value of a college degree | News, Sports, Jobs - The Express", - "type": "perspectives", - "url": "https://www.lockhaven.com/opinion/columns/2026/02/the-value-of-a-college-degree/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 20, - "sub_rank": 11, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Most Americans Don’t Value 4-Year Degree | The View", - "type": "perspectives", - "url": "https://www.youtube.com/watch?v=K-uiASmRWFo" - }, - { - "cite": "https://www.knox.edu › spring-2018 › features › yes-c...", - "cmpt_rank": 8, - "details": null, - "error": null, - "section": "main", - "serp_rank": 21, - "sub_rank": 0, - "sub_type": null, - "text": "On average, college graduates earn $1 million more over their lifetimes than high school graduates . A recent study by the Brookings Institution further shows ... Read more", - "title": "Yes, College is Worth It: Busting Myths about Higher ...", - "type": "general", - "url": "https://www.knox.edu/magazine/spring-2018/features/yes-college-is-worth-it" - }, - { - "cite": "https://magazine.utah.edu › issues › fall-2024 › is-colleg...", - "cmpt_rank": 9, - "details": null, - "error": null, - "section": "main", - "serp_rank": 22, - "sub_rank": 0, - "sub_type": null, - "text": "“The rewards are clear— college graduates see a substantial income boost, lower rates of poverty and unemployment, and greater confidence about the future,” ... Read more", - "title": "Is College Still Worth It?", - "type": "general", - "url": "https://magazine.utah.edu/issues/fall-2024/is-college-still-worth-it/" - }, - { - "cite": "https://freopp.org › whitepapers › is-college-worth-it-a-...", - "cmpt_rank": 10, - "details": null, - "error": null, - "section": "main", - "serp_rank": 23, - "sub_rank": 0, - "sub_type": null, - "text": "Median ROI drops from $306,000 before the completion adjustment to just $129,000 after the adjustment. These results suggest that college is still a good bet on ... Read more", - "title": "Is College Worth It? A Comprehensive Return on ...", - "type": "general", - "url": "https://freopp.org/whitepapers/is-college-worth-it-a-comprehensive-return-on-investment-analysis/" - }, - { - "cite": "https://www.wsj.com › Opinion › Future View", - "cmpt_rank": 11, - "details": null, - "error": null, - "section": "main", - "serp_rank": 24, - "sub_rank": 0, - "sub_type": null, - "text": "Jan 6, 2026 — Higher education still offers substantial returns for many . But its value depends on clarity of purpose—an understanding of what getting a ... Read more", - "title": "Is a College Degree Worth It?", - "type": "general", - "url": "https://www.wsj.com/opinion/is-a-college-degree-worth-it-f7753b32?gaa_at=eafs&gaa_n=AWEtsqfH-jZgoz1cYj29yDyc281msrq8junlzMpGL_eBI-vmMFXiNr5007Yp&gaa_ts=69868542&gaa_sig=rYCbArO3w7nf21W9d-pbNRCAGa83VL8-hbRryl7VLLLDNKmAwTyXy5zWf2_aqEKeyiy2fUpqRT7Pv5XlxRnG5w%3D%3D" - }, - { - "cite": null, - "cmpt_rank": 12, - "details": { - "heading": "People also search for", - "items": [ - "Is college worth it reddit", - "Is college worth it article", - "Is college worth it pros and cons", - "Is college worth it Essay", - "Is college worth it New York Times", - "Is college worth it scholarly articles", - "Is College Worth It book", - "Is College Worth It Pew Research" - ], - "type": "text" - }, - "error": null, - "section": "footer", - "serp_rank": 25, - "sub_rank": 0, - "sub_type": "people_also_search_for", - "text": "Is college worth it reddit<|>Is college worth it article<|>Is college worth it pros and cons<|>Is college worth it Essay<|>Is college worth it New York Times<|>Is college worth it scholarly articles<|>Is College Worth It book<|>Is College Worth It Pew Research", - "title": null, - "type": "searches_related", - "url": null - } - ] -} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[ad1eb5594673].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[ad1eb5594673].json new file mode 100644 index 00000000..e96cf992 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[ad1eb5594673].json @@ -0,0 +1,240 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": false, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 2300000.0, + "result_estimate_time": 0.19 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": null, + "error": null, + "section": "header", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "location_choose_area", + "text": null, + "title": "Results for Lower Manhattan, New York", + "type": "notice", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "sections": [ + { + "heading": "Investment banking layoffs", + "text": "Hundreds of jobs Barclays plans to cut hundreds of jobs in its investment banking division, research, and global markets. This is part of a long-term strategy to cut costs and increase profitability." + } + ], + "sources": [ + { + "favicon": null, + "publisher": "Business Standard", + "snippet": "Dec 28, 2024 — Barclays recently terminated 15 bankers and traders in New York just before the holidays, opting to issue termination ...", + "source_id": null, + "title": "Barclays fires 15 bankers, denies bonuses ahead of holidays; faces flak", + "url": "https://www.business-standard.com/companies/news/barclays-layoffs-ny-bankers-denies-bonuses-holidays-backlash-124122800280_1.html#:~:text=Barclays%20recently%20terminated%2015%20bankers,should%20not%20be%20considered%20discretionary." + }, + { + "favicon": null, + "publisher": "Bergen Record", + "snippet": "Jun 4, 2024 — “Banks are reducing back-office costs, and this includes people and head count reductions, unfortunately,” said Christo...", + "source_id": null, + "title": "Barclays cutting 70 North Jersey jobs as more layoffs hit financial sector", + "url": "https://www.northjersey.com/story/news/business/2024/06/04/barclays-laying-off-70-employees-in-north-jersey/73971878007/#:~:text=%E2%80%9CBanks%20are%20reducing%20back%2Doffice,in%20New%20Jersey%20this%20year." + }, + { + "favicon": null, + "publisher": "Times of India", + "snippet": "Dec 28, 2024 — A Barclays spokesperson justified the layoffs, saying, “We regularly review our talent pool to ensure that we are inve...", + "source_id": null, + "title": "Barclays fires 15 bankers before Christmas | World News", + "url": "https://timesofindia.indiatimes.com/world/us/barclays-fires-15-wall-street-bankers-before-christmas-holidays-axes-bonuses-in-brutal-layoff-move-report/articleshow/116729071.cms#:~:text=A%20Barclays%20spokesperson%20justified%20the,cost%2Dcutting%20with%20employee%20morale." + } + ], + "type": "ai_overview" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "sectioned", + "text": "Barclays has made several rounds of layoffs in recent months, including in its investment banking division, in New York, and in North Jersey . These layoffs are part of a long-term strategy to cut costs and increase profitability.", + "title": null, + "type": "ai_overview", + "url": null + }, + { + "cite": "https://www.efinancialcareers.com › news › barclays-is-c...", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "Jan 29, 2025 — Sources at the bank say Barclays yesterday cut 51 developers working across credit, equities, fixed income, FX and prime services technology in ...", + "title": "Barclays is cutting technology jobs and mandating extra ...", + "type": "general", + "url": "https://www.efinancialcareers.com/news/barclays-is-cutting-technology-jobs-and-mandating-extra-office-days" + }, + { + "cite": null, + "cmpt_rank": 3, + "details": { + "items": [ + "Is Barclays laying off employees?", + "Will Barclays layoff in 2024?", + "Is Barclays bank closing in 2024?", + "Is Barclays financially stable?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Is Barclays laying off employees?<|>Will Barclays layoff in 2024?<|>Is Barclays bank closing in 2024?<|>Is Barclays financially stable?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.reuters.com › business › finance › barclays...", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "May 1, 2024 — The bank axed around 5,000 jobs in 2023 after dealmaking activities slumped across the industry. In November, Reuters exclusively reported the ...", + "title": "Barclays moves to cut 'a few hundred' investment bank ...", + "type": "general", + "url": "https://www.reuters.com/business/finance/barclays-moves-cut-a-few-hundred-investment-bank-underperformers-2024-05-01/" + }, + { + "cite": "https://www.wallstreetoasis.com › investment-banking", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "Apr 30, 2024 — Least unexpected layoffs this year after the Citi & UBS ones. Barclay's big deal makers were the ex-Lehman guys... a quarter of whom just left ...", + "title": "Big Layoff at Barclays - 5/1/24", + "type": "general", + "url": "https://www.wallstreetoasis.com/forum/investment-banking/big-layoff-at-barclays-5124" + }, + { + "cite": "https://www.northjersey.com › business › 2024/06/04", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "Jun 4, 2024 — Barclays bank is laying off 70 employees in Whippany , the latest job cuts in New Jersey's financial services sector.", + "title": "Barclays laying off 70 in North Jersey", + "type": "general", + "url": "https://www.northjersey.com/story/news/business/2024/06/04/barclays-laying-off-70-employees-in-north-jersey/73971878007/" + }, + { + "cite": "https://www.efinancialcareers.com › news › barclays-lon...", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "May 2, 2024 — Barclays has announced its intention of cutting £188m in costs from its investment bank by 2026. There are likely to be more redundancies to ...", + "title": "Barclays' London bankers say one team was cut more than ...", + "type": "general", + "url": "https://www.efinancialcareers.com/news/barclays-london-bankers-say-one-team-was-cut-more-than-the-rest-yesterday" + }, + { + "cite": "https://www.themiddlemarket.com › news-analysis › ba...", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "May 1, 2024 — The cuts include trading, advisory, capital markets and the firm's international corporate bank.", + "title": "Barclays Begins Implementing Job Cuts Across Investment Bank", + "type": "general", + "url": "https://www.themiddlemarket.com/news-analysis/barclays-begins-implementing-job-cuts-across-investment-bank" + }, + { + "cite": "https://www.bloomberg.com › news › articles › barclays-...", + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "May 1, 2024 — Barclays Plc has begun cutting hundreds of jobs as the firm embarks on a £2 billion cost-cutting drive that's part of Chief Executive ...", + "title": "Barclays Begins Implementing Job Cuts Across Investment ...", + "type": "general", + "url": "https://www.bloomberg.com/news/articles/2024-05-01/barclays-begins-implementing-job-cuts-across-investment-bank" + }, + { + "cite": "https://www.theguardian.com › business › feb › barclay...", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Feb 20, 2024 — Barclays to cut costs by £2bn , raising fears of further job losses. Bank announces drop in profits to £6.6bn as it says it plans to return more to shareholders.", + "title": "Barclays to cut costs by £2bn, raising fears of further job ...", + "type": "general", + "url": "https://www.theguardian.com/business/2024/feb/20/barclays-cut-costs-job-losses-profits" + }, + { + "cite": "https://www.cityam.com › barclays-starts-cutting-hundr...", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "May 1, 2024 — Barclays has reportedly started cutting hundreds of roles , including at its investment bank, as the group looks to cut costs and improve its share price.", + "title": "Barclays starts to cut hundreds of jobs in cost-saving bid ...", + "type": "general", + "url": "https://www.cityam.com/barclays-starts-cutting-hundreds-of-jobs-in-cost-saving-push-including-in-investment-bank/" + }, + { + "cite": "https://finance.yahoo.com › news › barclays-bcs-initiate...", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "May 3, 2024 — Barclays PLC BCS has commenced job cuts across investment banking (IB) and research division, per people familiar with the matter.", + "title": "Barclays (BCS) Initiates Job Cuts Across IB & Research Units", + "type": "general", + "url": "https://finance.yahoo.com/news/barclays-bcs-initiates-job-cuts-153300524.html" + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[b938dae25618].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[b938dae25618].json new file mode 100644 index 00000000..0f30a0e2 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[b938dae25618].json @@ -0,0 +1,282 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 13900000.0, + "result_estimate_time": 0.3 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "duration": "15 min", + "ingredients": "Pork sausage crumbles, buttermilk biscuits, flour, black pepper", + "n_reviews": "1.3K", + "rating": "4.8", + "source": "Allrecipes", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": "Easy Sausage Gravy and Biscuits", + "type": "recipes", + "url": "https://www.allrecipes.com/recipe/216391/easy-sausage-gravy-and-biscuits/" + }, + { + "cite": null, + "cmpt_rank": 0, + "details": { + "duration": "15 min", + "ingredients": "Pound pork sausage, buttermilk biscuits, red pepper flakes, butter, all purpose flour", + "n_reviews": "1.2K", + "rating": "5.0", + "source": "Tastes Better From Scratch", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 1, + "sub_type": null, + "text": null, + "title": "Sausage Gravy and Biscuits", + "type": "recipes", + "url": "https://tastesbetterfromscratch.com/sausage-gravy-biscuits/" + }, + { + "cite": null, + "cmpt_rank": 0, + "details": { + "duration": "25 min", + "ingredients": "Breakfast sausage, low fat, red pepper flakes, baking powder, sea salt", + "n_reviews": "17", + "rating": "4.9", + "source": "Natasha's Kitchen", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 2, + "sub_type": null, + "text": null, + "title": "Biscuits and Gravy Recipe", + "type": "recipes", + "url": "https://natashaskitchen.com/biscuits-and-gravy-recipe/" + }, + { + "cite": "https://www.allrecipes.com › ... › Breakfast and Brunch", + "cmpt_rank": 1, + "details": { + "rating": 5.0, + "scale": 5, + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "Dec 8, 2022 — This biscuits and gravy recipe uses jumbo buttermilk biscuits and pork sausage crumbles for a hearty, family-favorite breakfast that's ready ...", + "title": "Easy Sausage Gravy and Biscuits Recipe", + "type": "general", + "url": "https://www.allrecipes.com/recipe/216391/easy-sausage-gravy-and-biscuits/" + }, + { + "cite": "https://natashaskitchen.com › Breakfast", + "cmpt_rank": 2, + "details": { + "rating": 5.0, + "scale": 5, + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "Oct 24, 2023 — Our homemade Biscuits and Gravy Recipe is an easy-to-make classic Southern breakfast of creamy sausage gravy over warm buttery biscuits.", + "title": "Biscuits and Gravy Recipe", + "type": "general", + "url": "https://natashaskitchen.com/biscuits-and-gravy-recipe/" + }, + { + "cite": "https://tastesbetterfromscratch.com › sausage-gravy-bisc...", + "cmpt_rank": 3, + "details": { + "rating": 5.0, + "scale": 5, + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "Jul 19, 2020 — Ingredients. 1x 2x 3x · ▢ 8 Flakey Buttermilk Biscuits · ▢ 1 pound pork sausage · ▢ 2 Tablespoon + 1 teaspoon(s)s all-purpose flour · ▢ 2 1/2 ...", + "title": "The BEST Biscuits and Gravy recipe", + "type": "general", + "url": "https://tastesbetterfromscratch.com/sausage-gravy-biscuits/" + }, + { + "cite": null, + "cmpt_rank": 4, + "details": { + "items": [ + "What is the gravy in biscuits and gravy made of?", + "What can you add to sausage gravy to make it taste better?", + "Do you drain sausage for biscuits and gravy?", + "How do you fix bland biscuits and gravy?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "What is the gravy in biscuits and gravy made of?<|>What can you add to sausage gravy to make it taste better?<|>Do you drain sausage for biscuits and gravy?<|>How do you fix bland biscuits and gravy?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "70+ comments · 1 year ago", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "For gravy I get a pound tube of bob Evans original pork sausage (can't stand jimmy deans, but get whatever you like) brown it up, add 1/4 cup ...", + "title": "How to make the best Sausage gravy and biscuits : r/Cooking", + "type": "general", + "url": "https://www.reddit.com/r/Cooking/comments/zpy1a4/how_to_make_the_best_sausage_gravy_and_biscuits/" + }, + { + "cite": "https://sugarspunrun.com › Breakfast", + "cmpt_rank": 6, + "details": { + "rating": 5.0, + "scale": 5, + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "Sep 8, 2022 — An old-fashioned homemade sausage gravy recipe , this one is flavorful and foolproof. Serve over my award-winning (and easy!) biscuit recipe .", + "title": "Homemade Sausage Gravy Recipe", + "type": "general", + "url": "https://sugarspunrun.com/sausage-gravy/" + }, + { + "cite": "https://www.browneyedbaker.com › buttermilk-biscuits...", + "cmpt_rank": 7, + "details": { + "rating": 5.0, + "scale": 5, + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Aug 12, 2023 — For the Biscuits : · 2½ cups (300 g) all-purpose flour · 2 tablespoons baking powder · 2 teaspoons granulated sugar · 1 teaspoon salt · ½ cup (113 g) ...", + "title": "Southern-Style Biscuits and Gravy", + "type": "general", + "url": "https://www.browneyedbaker.com/buttermilk-biscuits-with-sausage-gravy/" + }, + { + "cite": "https://www.blessthismessplease.com › ... › Breakfast", + "cmpt_rank": 8, + "details": { + "rating": 5.0, + "scale": 5, + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "Oct 28, 2022 — For the Biscuits · 2 cups all-purpose flour · 4 teaspoons baking powder · 3 tablespoons white sugar · 1/2 teaspoon salt · 1/2 teaspoon cream of ...", + "title": "Biscuits & Gravy | The Easy Recipe I've Been Making For ...", + "type": "general", + "url": "https://www.blessthismessplease.com/homemade-biscuits-and-gravy/" + }, + { + "cite": "104.4K+ views · 3 months ago", + "cmpt_rank": 9, + "details": { + "duration": "3:11", + "source": null, + "type": "video" + }, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": "video", + "text": "...biscuitor two into bite-size pieces and place on a plate. Spoon over ... TikTokBiscuits and GravyCasseroleRecipe. Mommy Mouse Clubhouse ...", + "title": "Ree Drummond's Biscuits and Sausage Pepper Gravy", + "type": "general", + "url": "https://www.youtube.com/watch?v=Dg8OZF5masQ" + }, + { + "cite": "https://www.quora.com › What-is-a-good-way-to-make...", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": null, + "text": "Jun 5, 2019 — Use a combination of sausage and bacon drippings. Add an equal amount of flour to the skillet and cook for a minute or two. Slowly whisk in ...", + "title": "What is a good way to make biscuit gravy from scratch?", + "type": "general", + "url": "https://www.quora.com/What-is-a-good-way-to-make-biscuit-gravy-from-scratch" + }, + { + "cite": null, + "cmpt_rank": 11, + "details": { + "heading": "Related searches", + "items": [ + "pioneer woman biscuits and gravy", + "biscuits and gravy without sausage", + "biscuits and gravy near me", + "biscuits and gravy ingredients", + "old fashioned sausage gravy", + "Sausage gravy recipe", + "Easy biscuits and gravy recipe", + "Biscuit recipe" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "pioneer woman biscuits and gravy<|>biscuits and gravy without sausage<|>biscuits and gravy near me<|>biscuits and gravy ingredients<|>old fashioned sausage gravy<|>Sausage gravy recipe<|>Easy biscuits and gravy recipe<|>Biscuit recipe", + "title": null, + "type": "searches_related", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[bb50002482cc].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[bb50002482cc].json new file mode 100644 index 00000000..fd3948eb --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[bb50002482cc].json @@ -0,0 +1,367 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": false, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 208000000.0, + "result_estimate_time": 0.42 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "sections": [ + { + "heading": "Counties included:", + "text": "Los Angeles County : The center of the metropolitan area Orange County : Located to the southeast Riverside County : Located to the east" + } + ], + "sources": [ + { + "favicon": null, + "publisher": "Wikipedia", + "snippet": "Greater Los Angeles is the most populous metropolitan area in the U.S. state of California, encompassing five counties in Southern...", + "source_id": null, + "title": "Greater Los Angeles - Wikipedia", + "url": "https://en.wikipedia.org/wiki/Greater_Los_Angeles#:~:text=Greater%20Los%20Angeles%20is%20the,largest%20megacities%20in%20the%20world." + }, + { + "favicon": null, + "publisher": "Wikipedia", + "snippet": "Los Angeles, often referred to by its initials L.A., is the most populous city in the U.S. state of California. With an estimated ...", + "source_id": null, + "title": "Los Angeles - Wikipedia", + "url": "https://en.wikipedia.org/wiki/Los_Angeles#:~:text=Los%20Angeles%2C%20often%20referred%20to,May%2023%2C%201835" + }, + { + "favicon": null, + "publisher": "Wikipedia", + "snippet": "The base fare for Metro services, including local and express buses and Metro Rail, is $1.75. Metro introduced daily and weekly fa...", + "source_id": null, + "title": "Los Angeles County Metropolitan Transportation Authority - Wikipedia", + "url": "https://en.wikipedia.org/wiki/Los_Angeles_County_Metropolitan_Transportation_Authority#:~:text=The%20base%20fare%20for%20Metro,its%20cost%20and%20perceived%20ineffectiveness." + } + ], + "type": "ai_overview" + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "sectioned", + "text": "The Los Angeles metropolitan area, also known as the Greater Los Angeles Area, is a region in Southern California that includes Los Angeles County and the surrounding counties . It's the second-largest metropolitan area in the United States, after New York City.", + "title": null, + "type": "ai_overview", + "url": null + }, + { + "cite": "https://en.wikipedia.org › wiki › Greater_Los_Angeles", + "cmpt_rank": 1, + "details": null, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": null, + "text": "Greater Los Angeles is the most populous metropolitan area in the US state of California, encompassing five counties in Southern California", + "title": "Greater Los Angeles", + "type": "general", + "url": "https://en.wikipedia.org/wiki/Greater_Los_Angeles" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": { + "items": [ + "What is considered a metropolitan area?", + "What counties are in the Los Angeles metro area?", + "What is Los Angeles metropolitan area code?", + "What are the metropolitan areas of Southern California?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": null, + "text": "What is considered a metropolitan area?<|>What counties are in the Los Angeles metro area?<|>What is Los Angeles metropolitan area code?<|>What are the metropolitan areas of Southern California?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.carolmendelmaps.com › mapla", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": null, + "text": "The map features the entire metropolitan area , from San Fernando and the San Gabriel mountains in the north to Newport Beach and Irvine in the south.", + "title": "Los Angeles metropolitan area map", + "type": "general", + "url": "https://www.carolmendelmaps.com/mapla/la1.html" + }, + { + "cite": "https://simple.wikipedia.org › wiki › Greater_Los_Ange...", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 0, + "sub_type": null, + "text": "The Los Angeles metropolitan area has a total area of 4,850 square miles (12,561.442 km 2 ). The wider combined statistical area covers 33,954 square miles (87, ...", + "title": "Greater Los Angeles Area - Simple English Wikipedia, the ...", + "type": "general", + "url": "https://simple.wikipedia.org/wiki/Greater_Los_Angeles_Area" + }, + { + "cite": "https://labormarketinfo.edd.ca.gov › definitions › metro...", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 0, + "sub_type": null, + "text": "The MSA and their Metropolitan Divisions are: Los Angeles-Long Beach-Anaheim, CA MSA ; Los Angeles-Long Beach-Glendale, CA MD (Los Angeles County); Anaheim ...", + "title": "Metropolitan Statistical Areas in California", + "type": "general", + "url": "https://labormarketinfo.edd.ca.gov/definitions/metropolitan-statistical-areas.html" + }, + { + "cite": "https://www.whittier.edu › about › area", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 0, + "sub_type": null, + "text": "The Los Angeles area (the five-county metropolitan area composed of Los Angeles, Orange, Riverside, San Bernardino, and Ventura Counties) is the 11th-largest ...", + "title": "Greater Los Angeles Area", + "type": "general", + "url": "https://www.whittier.edu/about/area" + }, + { + "cite": "https://metropolitan.fandom.com › wiki › Greater_Los_...", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "The Greater Los Angeles area is generally taken to include the Los Angeles metropolitan area , the Inland Empire, and Ventura County.", + "title": "Greater Los Angeles Area | Metropolitan Wiki - Fandom", + "type": "general", + "url": "https://metropolitan.fandom.com/wiki/Greater_Los_Angeles_Area" + }, + { + "cite": "70+ comments · 1 year ago", + "cmpt_rank": 8, + "details": { + "items": [ + { + "text": "Does \"Los Angeles\" usually refer to Greater Los ... - Reddit", + "url": "https://www.reddit.com/r/AskLosAngeles/comments/17829w5/does_los_angeles_usually_refer_to_greater_los/" + }, + { + "text": "The Los Angeles metropolitan area [3850 x 2311] [OS] - Reddit", + "url": "https://www.reddit.com/r/MapPorn/comments/4gv7d6/the_los_angeles_metropolitan_area_3850_x_2311_os/" + }, + { + "text": "Is the Inland Empire a part of the Los Angeles Metropolitan ...", + "url": "https://www.reddit.com/r/InlandEmpire/comments/12kpdag/is_the_inland_empire_a_part_of_the_los_angeles/" + }, + { + "text": "Los Angeles is a city surrounded by 88 different suburbs - Reddit", + "url": "https://www.reddit.com/r/geography/comments/1bjhj2o/los_angeles_is_a_city_surrounded_by_88_different/" + }, + { + "text": "More results from www.reddit.com", + "url": "/search?q=metropolitan+los+angeles+area+site:www.reddit.com&sca_esv=14324ad49b0ab5cb&sa=X&ved=2ahUKEwig57rU-6-LAxV2klYBHTraOTAQrQIoBHoECFQQBQ" + } + ], + "type": "hyperlinks" + }, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": "submenu", + "text": "It is 1,740 square miles now , larger than than the state of Rhode Island.", + "title": "Are you surprised/amazed with just how large and ...", + "type": "general", + "url": "https://www.reddit.com/r/AskLosAngeles/comments/18yu2u9/are_you_surprisedamazed_with_just_how_large_and/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": "medium", + "text": null, + "title": "Los Angeles metropolitan area map", + "type": "images", + "url": "https://www.carolmendelmaps.com/mapla/la1.html" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 1, + "sub_type": "medium", + "text": null, + "title": "The Los Angeles metropolitan area [3850 x 2311] [OS] : r/MapPorn", + "type": "images", + "url": "https://www.reddit.com/r/MapPorn/comments/4gv7d6/the_los_angeles_metropolitan_area_3850_x_2311_os/" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 2, + "sub_type": "medium", + "text": null, + "title": "Greater Los Angeles | Familypedia | Fandom", + "type": "images", + "url": "https://familypedia.fandom.com/wiki/Greater_Los_Angeles" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 3, + "sub_type": "medium", + "text": null, + "title": "Los Angeles metropolitan area map", + "type": "images", + "url": "https://www.carolmendelmaps.com/mapla/la1.html" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 4, + "sub_type": "medium", + "text": null, + "title": "Greater Los Angeles - Wikipedia", + "type": "images", + "url": "https://en.wikipedia.org/wiki/Greater_Los_Angeles" + }, + { + "cite": null, + "cmpt_rank": 9, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 5, + "sub_type": "medium", + "text": null, + "title": "Los Angeles-Long Beach-Anaheim, CA MSA Situation & Outlook ...", + "type": "images", + "url": "https://proximityone.com/metros/2013/cbsa31080.htm" + }, + { + "cite": "https://support.crunchbase.com › en-us › articles › 360...", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": null, + "text": "Feb 4, 2021 — Here's the current breakdown of the Greater Los Angeles Area region: Los Angeles; Santa Monica; Irvine; Beverly Hills; Lucerne Valley; Pasadena  ...", + "title": "What cities are in the Greater Los Angeles region?", + "type": "general", + "url": "https://support.crunchbase.com/hc/en-us/articles/360009895834-What-cities-are-in-the-Greater-Los-Angeles-region" + }, + { + "cite": "https://www.metro.net", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 0, + "sub_type": null, + "text": "Explore Los Angeles by public transit. Start with Metro schedules, maps, arrival times, parking info, trip planning, stations, and fares.", + "title": "Metro | Bus, Rail, Subway, Bike & Micro in Los Angeles", + "type": "general", + "url": "https://www.metro.net/" + }, + { + "cite": null, + "cmpt_rank": 12, + "details": { + "heading": "People also search for", + "items": [ + "Los Angeles", + "California", + "New York Metropolitan Area", + "Los Angeles County", + "Greater Los Angeles", + "San Francisco Bay Area", + "Metropolitan los angeles area map", + "Metropolitan los angeles area towns", + "Metropolitan los angeles area cities", + "Los Angeles County population", + "Greater Los Angeles map", + "Greater Los Angeles map with cities", + "Orange County population", + "La area map" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": "people_also_search_for", + "text": "Los Angeles<|>California<|>New York Metropolitan Area<|>Los Angeles County<|>Greater Los Angeles<|>San Francisco Bay Area<|>Metropolitan los angeles area map<|>Metropolitan los angeles area towns<|>Metropolitan los angeles area cities<|>Los Angeles County population<|>Greater Los Angeles map<|>Greater Los Angeles map with cities<|>Orange County population<|>La area map", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "rhs", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": "panel_rhs", + "text": null, + "title": "See results about", + "type": "knowledge", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[be99c971b8f7].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[be99c971b8f7].json deleted file mode 100644 index f17ab512..00000000 --- a/tests/__snapshots__/test_parse_serp/test_parse_serp[be99c971b8f7].json +++ /dev/null @@ -1,509 +0,0 @@ -{ - "features": { - "captcha": false, - "infinity_scroll": false, - "language": "en", - "main_layout": "standard", - "notice_no_results": false, - "notice_server_error": false, - "notice_shortened_query": false, - "overlay_precise_location": false, - "result_estimate_count": 2880000000.0, - "result_estimate_time": 0.35 - }, - "results": [ - { - "cite": null, - "cmpt_rank": 0, - "details": { - "citations": [ - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "1", - "2", - "14", - "6" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "13", - "15" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "4", - "8" - ] - } - ], - "sources": [ - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.weather.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Weather Service (.gov)", - "snippet": "While all colors are scattered by air molecules, violet and blue are scattered most. The sky looks blue, not violet, because our e...", - "source_id": "14", - "title": "Why Is The Sky Blue? - National Weather Service", - "url": "https://www.weather.gov/fgz/SkyBlue" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.britannica.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Britannica", - "snippet": "One of the perennial questions of childhood is “Why is the sky blue?” You may have asked this as a child, or you may have a child ...", - "source_id": "6", - "title": "Why Is the Sky Blue? | Britannica", - "url": "https://www.britannica.com/story/why-is-the-sky-blue" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.nesdis.noaa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Environmental Satellite, Data, and Information Service (.gov)", - "snippet": "The Short Answer. Gases and particles in Earth's atmosphere scatter sunlight in all directions. Blue light is scattered more than ...", - "source_id": "1", - "title": "Why Is the Sky Blue? | NESDIS", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://math.ucr.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "University of California, Riverside", - "snippet": "The sky is blue because molecules in the air scatter blue light from the sun more than they scatter red light. This is due to the ...", - "source_id": "2", - "title": "Why is the sky blue?", - "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.instagram.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Instagram", - "snippet": "Why Sun Changes Color Throughout the Day? 🌞 Sunset: red. Noon: white. What's happening? At noon, light travels straight down - 10...", - "source_id": "8", - "title": "Why is the sky blue? It's not. You might know that as sunlight ... - Instagram", - "url": "https://www.instagram.com/reel/DIZJjPvOtIN/" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.rmg.co.uk&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Royal Museums Greenwich", - "snippet": "This content is hosted by a third party. Please allow all cookies to watch the video. ... These different colours have different w...", - "source_id": "13", - "title": "Why is the sky blue? | Royal Observatory", - "url": "https://www.rmg.co.uk/stories/space-astronomy/why-sky-blue" - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.facebook.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Facebook", - "snippet": "The sky appears blue because of a phenomenon called Rayleigh scattering, which is the scattering of light by small particles or mo...", - "source_id": "15", - "title": "Why is the sky blue? It's an age old question that actually has a very ...", - "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://www.wsav.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "WSAV-TV", - "snippet": "In summer, the sun sits higher in the sky, meaning sunlight has a shorter path through the atmosphere. In winter, the sun is lower...", - "source_id": "4", - "title": "Why the sky looks bluer in fall and winter - WSAV-TV", - "url": "https://www.wsav.com/weather-news/why-the-sky-looks-bluer-in-fall-and-winter/#:~:text=In%20summer%2C%20the%20sun%20sits%20higher%20in,intensifying%20the%20blue%20appearance%20of%20the%20sky." - } - ], - "type": "ai_overview" - }, - "error": null, - "section": "main", - "serp_rank": 0, - "sub_rank": 0, - "sub_type": "flat", - "text": "The sky is blue due to Rayleigh scattering , a phenomenon where Earth's atmospheric molecules scatter blue light more effectively than other colors because blue light has shorter, smaller waves. Sunlight, composed of all colors, is scattered by gases like nitrogen and oxygen in the atmosphere; shorter blue wavelengths are scattered in all directions, saturating the sky with a blue appearance, though our eyes are more sensitive to blue than violet. The Role of Sunlight and Atmospheric Molecules Sunlight Composition : The white light from the sun is a mixture of all colors in the visible spectrum, each with different wavelengths. Atmospheric Scattering : When sunlight enters Earth's atmosphere, it interacts with gas molecules (primarily nitrogen and oxygen). Wavelength and Scattering : Shorter wavelengths of light, like blue and violet, are scattered much more strongly than longer wavelengths, such as red and orange. Why We See Blue, Not Violet Violet Scattering : Violet light has an even shorter wavelength than blue light and is scattered more. Eye Sensitivity : However, the sky appears blue rather than violet because human eyes are more sensitive to blue light. Sun's Emission : The sun also emits more energy as blue light than as violet light, contributing to the blue appearance. Factors Affecting Sky Color Atmospheric Path Length : Sunlight has a shorter path through the atmosphere at noon, and at sunset, it travels a longer path through more air. Sky Color at Sunset/Sunrise : As the path length increases, more blue light is scattered away, allowing the longer, reddish wavelengths to pass through, creating the reddish-orange colors of sunsets. High Altitudes : At higher altitudes, there are fewer atmospheric molecules, so less light is scattered. Space : In space, where there are virtually no molecules to scatter light, the sky appears black.", - "title": null, - "type": "ai_overview", - "url": null - }, - { - "cite": null, - "cmpt_rank": 1, - "details": { - "items": [ - "Why is the sky blue short answer?", - "What is the true color of our sky?", - "How to explain to a kid why the sky is blue?", - "Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar" - ], - "type": "text" - }, - "error": null, - "section": "main", - "serp_rank": 1, - "sub_rank": 0, - "sub_type": null, - "text": "Why is the sky blue short answer?<|>What is the true color of our sky?<|>How to explain to a kid why the sky is blue?<|>Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar", - "title": null, - "type": "people_also_ask", - "url": null - }, - { - "cite": "https://spaceplace.nasa.gov › blue-sky", - "cmpt_rank": 2, - "details": null, - "error": null, - "section": "main", - "serp_rank": 2, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://spaceplace.nasa.gov/blue-sky/" - }, - { - "cite": "160+ comments · 2 years ago", - "cmpt_rank": 3, - "details": { - "items": [ - { - "text": "Eli5: why is the sky blue? : r/explainlikeimfive - Reddit", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1iytxes/eli5_why_is_the_sky_blue/" - }, - { - "text": "Why is the sky blue? Refraction. Why? Because ...", - "url": "https://www.reddit.com/r/askscience/comments/5tg0y2/why_is_the_sky_blue_refraction_why_because/" - }, - { - "text": "More results from www.reddit.com", - "url": "/search?q=why+is+the+sky+blue+site:www.reddit.com&sca_esv=a23961d46812ec7b&sa=X&ved=2ahUKEwj3_L-Hh8aSAxWwCBAIHZSuDXkQrQIoAnoECB8QAw" - } - ], - "type": "hyperlinks" - }, - "error": null, - "section": "main", - "serp_rank": 3, - "sub_rank": 0, - "sub_type": "submenu", - "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", - "title": "Why is the sky blue? Do I understand it correctly", - "type": "general", - "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" - }, - { - "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", - "cmpt_rank": 4, - "details": null, - "error": null, - "section": "main", - "serp_rank": 4, - "sub_rank": 0, - "sub_type": null, - "text": "Gases and particles in Earth's atmosphere scatter sunlight in all directions . Blue light is scattered more than other colors because it travels as shorter, ...", - "title": "Why Is the Sky Blue? - NESDIS - NOAA", - "type": "general", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" - }, - { - "cite": "https://math.ucr.edu › home › baez › physics › General", - "cmpt_rank": 5, - "details": null, - "error": null, - "section": "main", - "serp_rank": 5, - "sub_rank": 0, - "sub_type": null, - "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" - }, - { - "cite": "https://kids.nationalgeographic.com › books › article", - "cmpt_rank": 6, - "details": null, - "error": null, - "section": "main", - "serp_rank": 6, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more because of its short, choppy wavelength, making it the color we see the most. Read more", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://kids.nationalgeographic.com/books/article/sky" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 7, - "sub_rank": 0, - "sub_type": "what_people_are_saying", - "text": null, - "title": "We see the sky as blue, but in reality it is completely black. The blue color is simply an optical illusion caused by the scattering of sunlight in our atmosphere.", - "type": "perspectives", - "url": "https://www.facebook.com/groups/ScienceKiDuniya/posts/3440058569495988/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 8, - "sub_rank": 1, - "sub_type": "what_people_are_saying", - "text": null, - "title": "The REAL Reason the Sky Looks Blue", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 9, - "sub_rank": 2, - "sub_type": "what_people_are_saying", - "text": null, - "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", - "type": "perspectives", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 10, - "sub_rank": 3, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Sky Blue?", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/n4ElQOE3lyI" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 11, - "sub_rank": 4, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Sky Blue? Credit to Discover.", - "type": "perspectives", - "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 12, - "sub_rank": 5, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why Is the Sky Blue? Science Behind the Blue Sky\n\n#WhyIsTheSkyBlue\n#BlueSky\n#ScienceExplained\n#RayleighScattering\n#spacescience", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DT7ufKNj70z/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 13, - "sub_rank": 6, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 14, - "sub_rank": 7, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Night Sky Black and not Blue? | February 2 - February 8 | Star Gazers", - "type": "perspectives", - "url": "https://www.youtube.com/watch?v=pIZBeINyK8w" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 15, - "sub_rank": 8, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", - "type": "perspectives", - "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 16, - "sub_rank": 9, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why the sky is blue", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DSOGSiPkTjw/?hl=en" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 17, - "sub_rank": 10, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/o1pTCw8cg10" - }, - { - "cite": null, - "cmpt_rank": 7, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 18, - "sub_rank": 11, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why are #sunsets red but the #sky is blue? They are both caused by the same thing. \n\nThe light from the sun is white-ish. That means it's a combination of red, green and blue light. It turns out that the blue light is scattered by the air (nitrogen and oxygen molecules). This scattered blue light enters your eyes and you see the sky as blue.\n\nFor the sunset, the sun is low in the sky and passes through MORE air and that means that MORE of the blue light has been scattered. With less blue light from the sun, it looks more red.\n\nHere's a great physics demo. Put some powdered coffee creamer in water and shine a flashlight through it. If you look at the water from the side, it looks blueish. Looking into the flashlight it looks more red. Just like the sunset and sky.\n\n#physics #science #stem #physicsdemo", - "type": "perspectives", - "url": "https://www.instagram.com/p/DR2aqixgJUa/" - }, - { - "cite": "865.1K+ views · 5 years ago", - "cmpt_rank": 8, - "details": null, - "error": null, - "section": "main", - "serp_rank": 19, - "sub_rank": 0, - "sub_type": "video", - "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" - }, - { - "cite": "50+ reactions · 1 year ago", - "cmpt_rank": 9, - "details": null, - "error": null, - "section": "main", - "serp_rank": 20, - "sub_rank": 0, - "sub_type": "video", - "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", - "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", - "type": "general", - "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" - }, - { - "cite": "https://www.uu.edu › dept › physics › scienceguys", - "cmpt_rank": 10, - "details": null, - "error": null, - "section": "main", - "serp_rank": 21, - "sub_rank": 0, - "sub_type": null, - "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", - "title": "Why is the sky blue on Earth, but black in space or ...", - "type": "general", - "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" - }, - { - "cite": "https://www.optics4kids.org › what-is-optics › scattering", - "cmpt_rank": 11, - "details": null, - "error": null, - "section": "main", - "serp_rank": 22, - "sub_rank": 0, - "sub_type": null, - "text": "The primary cause of a blue sky and orange/red sunsets or sunrises is scattering by the gas molecules that make up our atmosphere . Read more", - "title": "Why is the sky blue? Why are sunsets red?", - "type": "general", - "url": "https://www.optics4kids.org/what-is-optics/scattering/why-is-the-sky-blue-why-are-sunsets-red" - }, - { - "cite": null, - "cmpt_rank": 12, - "details": { - "heading": "People also search for", - "items": [ - "Why is the sky blue Class 10", - "Why is the sky blue for kids", - "Why is the sky blue no ai", - "Why is the sky blue reflection of ocean", - "Why is the sky blue and not violet", - "Why is the sky blue Army", - "Why is the sky blue long answer", - "Why is the sky blue essay" - ], - "type": "text" - }, - "error": null, - "section": "footer", - "serp_rank": 23, - "sub_rank": 0, - "sub_type": "people_also_search_for", - "text": "Why is the sky blue Class 10<|>Why is the sky blue for kids<|>Why is the sky blue no ai<|>Why is the sky blue reflection of ocean<|>Why is the sky blue and not violet<|>Why is the sky blue Army<|>Why is the sky blue long answer<|>Why is the sky blue essay", - "title": null, - "type": "searches_related", - "url": null - } - ] -} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[c9ab650f5bda].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[c9ab650f5bda].json deleted file mode 100644 index b0800373..00000000 --- a/tests/__snapshots__/test_parse_serp/test_parse_serp[c9ab650f5bda].json +++ /dev/null @@ -1,530 +0,0 @@ -{ - "features": { - "captcha": false, - "infinity_scroll": false, - "language": "en", - "main_layout": "standard", - "notice_no_results": false, - "notice_server_error": false, - "notice_shortened_query": false, - "overlay_precise_location": false, - "result_estimate_count": 2980000000.0, - "result_estimate_time": 0.3 - }, - "results": [ - { - "cite": null, - "cmpt_rank": 0, - "details": { - "citations": [ - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "1", - "3", - "4", - "2", - "7" - ] - }, - { - "additional_count": 0, - "publisher": null, - "source_ids": [ - "6", - "12", - "15", - "8", - "5", - "9" - ] - } - ], - "sources": [ - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.weather.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Weather Service (.gov)", - "snippet": "While all colors are scattered by air molecules, violet and blue are scattered most. The sky looks blue, not violet, because our e...", - "source_id": "2", - "title": "Why Is The Sky Blue? - National Weather Service", - "url": "https://www.weather.gov/fgz/SkyBlue" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.nesdis.noaa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Environmental Satellite, Data, and Information Service NESDIS (.gov)", - "snippet": "Gases and particles in Earth's atmosphere scatter sunlight in all directions. Blue light is scattered more than other colors becau...", - "source_id": "1", - "title": "Why Is the Sky Blue? - NESDIS - NOAA", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue#:~:text=Gases%20and%20particles%20in%20Earth's%20atmosphere%20scatter,a%20blue%20sky%20most%20of%20the%20time." - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.facebook.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Facebook", - "snippet": "The sky appears blue because of a phenomenon called Rayleigh scattering. Sunlight is made up of all the colors of the rainbow, eac...", - "source_id": "4", - "title": "Why is the sky blue? It's an age old question that actually has a very simple answer The sun emits a white light that consists of all the colors of the rainbow At about 18 miles above the earth the light begins to interact with air molecules Those air molecules are the perfect size to scatter the blue light wavelengths, but all other colors continue down to the surface The blue light is scattered from molecule to molecule and reflected at you And now you know 🙃", - "url": "https://www.facebook.com/jakedunnekwch/posts/why-is-the-sky-blue-its-an-age-old-question-that-actually-has-a-very-simple-answ/1104044168200325/" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://morgridge.org&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Morgridge Institute for Research -", - "snippet": "The sky appears blue because of the scattering of blue light by the oxygen and nitrogen in our atmosphere. Light travels in waves,", - "source_id": "7", - "title": "Why is the sky blue? - Morgridge Institute for Research", - "url": "https://morgridge.org/blue-sky/why-is-the-sky-blue/" - }, - { - "favicon": "https://encrypted-tbn2.gstatic.com/faviconV2?url=https://www.reddit.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Reddit", - "snippet": "The sky is blue because blue light gets scattered the most during the day. Rayleigh scattering states that the amount of scatterin...", - "source_id": "8", - "title": "Why is the sky blue? Do I understand it correctly: : r/askscience", - "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://math.ucr.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "University of California, Riverside", - "snippet": "We have three types of colour receptors, or cones, in our retina. They are called red, blue and green because they respond most st...", - "source_id": "12", - "title": "Why is the sky blue? - UCR Math", - "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html#:~:text=We%20have%20three%20types%20of%20colour%20receptors%2C,visual%20system%20constructs%20the%20colours%20we%20see." - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://kids.nationalgeographic.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "National Geographic Kids", - "snippet": "The sky is blue because of the scattering of light in the atmosphere. The sun's light is made up of all the colors of the rainbow,", - "source_id": "6", - "title": "Why is the sky blue? | National Geographic Kids", - "url": "https://kids.nationalgeographic.com/books/article/sky" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.uu.edu&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Union University", - "snippet": "The sky appears blue because of light scattering. Light waves interact with air molecules, and the air particles quickly re-radiat...", - "source_id": "5", - "title": "Why is the sky blue on Earth, but black in space or on the Moon? | Science Guys | Union University, a Christian College in Tennessee", - "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.britannica.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Britannica", - "snippet": "One of the perennial questions of childhood is “Why is the sky blue?” You may have asked this as a child, or you may have a child ...", - "source_id": "15", - "title": "Why Is the Sky Blue? | Britannica", - "url": "https://www.britannica.com/story/why-is-the-sky-blue" - }, - { - "favicon": "https://encrypted-tbn0.gstatic.com/faviconV2?url=https://www.montrealsciencecentre.com&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "Montreal Science Centre", - "snippet": "Wrapped around the Earth, the atmosphere is made up of a thin layer of gases, mainly nitrogen and oxygen. This is where the air we...", - "source_id": "9", - "title": "Why is the Sky Blue? - Montreal Science Centre", - "url": "https://www.montrealsciencecentre.com/blog/why-the-sky-blue" - }, - { - "favicon": "https://encrypted-tbn1.gstatic.com/faviconV2?url=https://plus.nasa.gov&client=AIM&size=128&type=FAVICON&fallback_opts=TYPE,SIZE,URL", - "publisher": "NASA+ (.gov)", - "snippet": "When sunlight enters Earth's atmosphere, it encounters gases that scatter the light in all directions. The shorter, choppier waves...", - "source_id": "3", - "title": "Space Place in a Snap: Why Is the Sky Blue?", - "url": "https://plus.nasa.gov/video/space-place-in-a-snap-why-is-the-sky-blue-2/#:~:text=When%20sunlight%20enters%20Earth's%20atmosphere%2C%20it%20encounters,sky%20appears%20blue%20on%20a%20sunny%20day." - } - ], - "type": "ai_overview" - }, - "error": null, - "section": "main", - "serp_rank": 0, - "sub_rank": 0, - "sub_type": "flat", - "text": "The sky appears blue due to a phenomenon called Rayleigh scattering , where molecules and small particles in Earth's atmosphere scatter shorter, blue-wavelength sunlight in all directions more efficiently than other colors. Because this scattered blue light is distributed across the sky, our eyes perceive it as blue during the day. Key Details About Sky Color: Sunlight Composition: Sunlight appears white but is actually made up of all the colors of the rainbow. Rayleigh Scattering: As sunlight hits gas molecules (mostly nitrogen and oxygen) in the atmosphere, the light scatters. Blue light travels in smaller, shorter waves, causing it to scatter roughly four times more than red light . Why Not Violet? While violet light has an even shorter wavelength, the sky looks blue because the sun emits more blue light than violet, and human eyes are more sensitive to blue light. Sunsets/Sunrises: When the sun is low on the horizon, light passes through more atmosphere. This causes the blue light to scatter away entirely, leaving the longer red, yellow, and orange wavelengths to reach our eyes. Space is Black: Because space lacks an atmosphere and particles to scatter sunlight, the sky appears black, as seen from the Moon. The sky appears paler near the horizon because the scattered light must pass through more air, allowing more of the blue light to be scattered away before reaching the eyes.", - "title": null, - "type": "ai_overview", - "url": null - }, - { - "cite": null, - "cmpt_rank": 1, - "details": { - "items": [ - "Why is the sky blue short answer?", - "What is the true color of our sky?", - "How to explain to a kid why the sky is blue?", - "Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar" - ], - "type": "text" - }, - "error": null, - "section": "main", - "serp_rank": 1, - "sub_rank": 0, - "sub_type": null, - "text": "Why is the sky blue short answer?<|>What is the true color of our sky?<|>How to explain to a kid why the sky is blue?<|>Why is the Sky Blue? - YouTube YouTube https://www.youtube.com · Patristic Nectar YouTube https://www.youtube.com · Patristic Nectar", - "title": null, - "type": "people_also_ask", - "url": null - }, - { - "cite": "https://spaceplace.nasa.gov › blue-sky", - "cmpt_rank": 2, - "details": null, - "error": null, - "section": "main", - "serp_rank": 2, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more than the other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time. Read more", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://spaceplace.nasa.gov/blue-sky/" - }, - { - "cite": "160+ comments · 2 years ago", - "cmpt_rank": 3, - "details": { - "items": [ - { - "text": "Eli5: why is the sky blue? : r/explainlikeimfive - Reddit", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1iytxes/eli5_why_is_the_sky_blue/" - }, - { - "text": "I know why the sky is blue but why is the light that ...", - "url": "https://www.reddit.com/r/askscience/comments/1vcwur/i_know_why_the_sky_is_blue_but_why_is_the_light/" - }, - { - "text": "More results from www.reddit.com", - "url": "/search?q=why+is+the+sky+blue?+site:www.reddit.com&sca_esv=6e2c9a71c05a9bf4&sa=X&ved=2ahUKEwjCo7718sOSAxXZi7AFHSGSC98QrQIoAnoECB8QAw" - } - ], - "type": "hyperlinks" - }, - "error": null, - "section": "main", - "serp_rank": 3, - "sub_rank": 0, - "sub_type": "submenu", - "text": "Because the sun is intense, the scattering gives you lots of blueness in the sky . But because the sun doesn't travel very far through the ... Read more", - "title": "Why is the sky blue? Do I understand it correctly", - "type": "general", - "url": "https://www.reddit.com/r/askscience/comments/14566ig/why_is_the_sky_blue_do_i_understand_it_correctly/" - }, - { - "cite": "https://kids.nationalgeographic.com › books › article", - "cmpt_rank": 4, - "details": null, - "error": null, - "section": "main", - "serp_rank": 4, - "sub_rank": 0, - "sub_type": null, - "text": "Slowly, over the next two billion years, oxygen in the atmosphere rose to its present levels , and the sky took on the blue hue on view today.", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://kids.nationalgeographic.com/books/article/sky" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 5, - "sub_rank": 0, - "sub_type": "what_people_are_saying", - "text": null, - "title": "We see the sky as blue, but in reality it is completely black. The blue color is simply an optical illusion caused by the scattering of sunlight in our atmosphere.", - "type": "perspectives", - "url": "https://www.facebook.com/groups/ScienceKiDuniya/posts/3440058569495988/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 6, - "sub_rank": 1, - "sub_type": "what_people_are_saying", - "text": null, - "title": "The REAL Reason the Sky Looks Blue", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/HZTGNh6pVXY" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 7, - "sub_rank": 2, - "sub_type": "what_people_are_saying", - "text": null, - "title": "ELI5: Why does the sky appear blue even tho violet has a shorter wavelength than blue?", - "type": "perspectives", - "url": "https://www.reddit.com/r/explainlikeimfive/comments/1pt3rb6/eli5_why_does_the_sky_appear_blue_even_tho_violet/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 8, - "sub_rank": 3, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Sky Blue?", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/n4ElQOE3lyI" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 9, - "sub_rank": 4, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Sky Blue? Credit to Discover.", - "type": "perspectives", - "url": "https://www.facebook.com/ceressoft/posts/why-is-the-sky-blue-credit-to-discover/1445048500957636/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 10, - "sub_rank": 5, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why Is the Sky Blue? Science Behind the Blue Sky\n\n#WhyIsTheSkyBlue\n#BlueSky\n#ScienceExplained\n#RayleighScattering\n#spacescience", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DT7ufKNj70z/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 11, - "sub_rank": 6, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why is the Night Sky Black and not Blue? | February 2 - February 8 | Star Gazers", - "type": "perspectives", - "url": "https://www.youtube.com/watch?v=pIZBeINyK8w" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 12, - "sub_rank": 7, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Did you know that the sky in GTA V is blue because it is a reference to the real life sky which is also blue", - "type": "perspectives", - "url": "https://www.reddit.com/r/GTAV/comments/1qtseyg/did_you_know_that_the_sky_in_gta_v_is_blue/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 13, - "sub_rank": 8, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🌤️ Wednesday Weather Wisdom: Why Is the Sky Blue?\n\nEver wonder why our sky shines in that beautiful shade of blue? It all comes down to sunlight and the way it interacts with our atmosphere.\n\nSunlight may look white, but it’s actually made up of many colors, each with different wavelengths. When sunlight enters Earth’s atmosphere, it bumps into tiny molecules of air. This causes a process called Rayleigh scattering — light being scattered in all different directions.\n\nHere’s the key:\n🔹 Blue light has shorter wavelengths,\n🔹 Red light has longer wavelengths.\n\nShorter wavelengths scatter much more easily, so blue light gets bounced around the sky in every direction. That’s why no matter where you look during the day, your eyes pick up more scattered blue light than any other color.\n\nAt sunrise and sunset, the sun’s light travels through more atmosphere, so the shorter blue wavelengths scatter out before they reach you. The longer reds and oranges make it through — giving us those gorgeous", - "type": "perspectives", - "url": "https://www.instagram.com/reel/DSHGQYkkUhM/?hl=en" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 14, - "sub_rank": 9, - "sub_type": "what_people_are_saying", - "text": null, - "title": "🔵 Blue Skies… Ever notice the January skies seem more blue? It’s not just you. The sky IS more blue in the winter months. 🤯 Here’s why: - Cold air is packed together (dense) - The thick Summer humidity is gone - There just aren’t many dust particles, pollen or", - "type": "perspectives", - "url": "https://www.facebook.com/GarrettNLewis/posts/-blue-skiesever-notice-the-january-skies-seem-more-blueits-not-just-youthe-sky-i/1425119545652808/" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 15, - "sub_rank": 10, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Can Sky become Green? | Why is our sky Blue? | #shorts", - "type": "perspectives", - "url": "https://www.youtube.com/shorts/o1pTCw8cg10" - }, - { - "cite": null, - "cmpt_rank": 5, - "details": { - "heading": "What people are saying" - }, - "error": null, - "section": "main", - "serp_rank": 16, - "sub_rank": 11, - "sub_type": "what_people_are_saying", - "text": null, - "title": "Why are #sunsets red but the #sky is blue? They are both caused by the same thing. \n\nThe light from the sun is white-ish. That means it's a combination of red, green and blue light. It turns out that the blue light is scattered by the air (nitrogen and oxygen molecules). This scattered blue light enters your eyes and you see the sky as blue.\n\nFor the sunset, the sun is low in the sky and passes through MORE air and that means that MORE of the blue light has been scattered. With less blue light from the sun, it looks more red.\n\nHere's a great physics demo. Put some powdered coffee creamer in water and shine a flashlight through it. If you look at the water from the side, it looks blueish. Looking into the flashlight it looks more red. Just like the sunset and sky.\n\n#physics #science #stem #physicsdemo", - "type": "perspectives", - "url": "https://www.instagram.com/p/DR2aqixgJUa/" - }, - { - "cite": "https://www.nesdis.noaa.gov › about › atmosphere › wh...", - "cmpt_rank": 6, - "details": null, - "error": null, - "section": "main", - "serp_rank": 17, - "sub_rank": 0, - "sub_type": null, - "text": "Blue light is scattered more than other colors because it travels as shorter, smaller waves. This is why we see a blue sky most of the time.", - "title": "Why Is the Sky Blue? - NESDIS - NOAA", - "type": "general", - "url": "https://www.nesdis.noaa.gov/about/k-12-education/atmosphere/why-the-sky-blue" - }, - { - "cite": "https://math.ucr.edu › home › baez › physics › General", - "cmpt_rank": 7, - "details": null, - "error": null, - "section": "main", - "serp_rank": 18, - "sub_rank": 0, - "sub_type": null, - "text": "A clear cloudless day-time sky is blue because molecules in the air scatter blue light from the Sun more than they scatter red light. Read more", - "title": "Why is the sky blue?", - "type": "general", - "url": "https://math.ucr.edu/home/baez/physics/General/BlueSky/blue_sky.html" - }, - { - "cite": "864K+ views · 5 years ago", - "cmpt_rank": 8, - "details": null, - "error": null, - "section": "main", - "serp_rank": 19, - "sub_rank": 0, - "sub_type": "video", - "text": "That blue color we see comes from sunlight hitting earth's atmosphere a layer of gases that gives us air to breathe.", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://www.youtube.com/watch?v=ehUIlhKhzDA" - }, - { - "cite": "50+ reactions · 1 year ago", - "cmpt_rank": 9, - "details": null, - "error": null, - "section": "main", - "serp_rank": 20, - "sub_rank": 0, - "sub_type": "video", - "text": "Bluehas short, bouncy waves that scatter more when sunlight hits the gas particles in our atmosphere, filling theskywithbluelight.", - "title": "Why is the sky blue? ☁️ Kids are pros at asking “simple ...", - "type": "general", - "url": "https://www.facebook.com/AstroKirsten/videos/why-is-the-sky-blue-%EF%B8%8F-kids-are-pros-at-asking-simple-questions-that-are-secretly/9100762216611478/" - }, - { - "cite": "https://www.britannica.com › ... › Matter & Energy", - "cmpt_rank": 10, - "details": null, - "error": null, - "section": "main", - "serp_rank": 21, - "sub_rank": 0, - "sub_type": null, - "text": "Jan 22, 2026 — The midday sky appears blue, rather than a combination of blue and violet, because our eyes are more sensitive to blue light than to violet light. Read more", - "title": "Why Is the Sky Blue?", - "type": "general", - "url": "https://www.britannica.com/story/why-is-the-sky-blue" - }, - { - "cite": "https://www.uu.edu › dept › physics › scienceguys", - "cmpt_rank": 11, - "details": null, - "error": null, - "section": "main", - "serp_rank": 22, - "sub_rank": 0, - "sub_type": null, - "text": "In space or on the Moon there is no atmosphere to scatter light . The light from the sun travels a straight line without scattering and all the colors stay ... Read more", - "title": "Why is the sky blue on Earth, but black in space or ...", - "type": "general", - "url": "https://www.uu.edu/dept/physics/scienceguys/2000Oct.cfm" - }, - { - "cite": null, - "cmpt_rank": 12, - "details": { - "heading": "People also search for", - "items": [ - "Why is the sky blue Class 10", - "Why is the sky blue for kids", - "Why is the sky blue no ai", - "Why is the sky blue reflection of ocean", - "Why is the sky blue and not violet", - "Why is the sky blue Army", - "Why is the sky blue long answer", - "Why is the sky blue essay" - ], - "type": "text" - }, - "error": null, - "section": "footer", - "serp_rank": 23, - "sub_rank": 0, - "sub_type": "people_also_search_for", - "text": "Why is the sky blue Class 10<|>Why is the sky blue for kids<|>Why is the sky blue no ai<|>Why is the sky blue reflection of ocean<|>Why is the sky blue and not violet<|>Why is the sky blue Army<|>Why is the sky blue long answer<|>Why is the sky blue essay", - "title": null, - "type": "searches_related", - "url": null - } - ] -} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[cad43c3268a8].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[cad43c3268a8].json deleted file mode 100644 index d0dc8151..00000000 --- a/tests/__snapshots__/test_parse_serp/test_parse_serp[cad43c3268a8].json +++ /dev/null @@ -1,285 +0,0 @@ -{ - "features": { - "captcha": false, - "infinity_scroll": false, - "language": "en", - "main_layout": "standard", - "notice_no_results": false, - "notice_server_error": false, - "notice_shortened_query": false, - "overlay_precise_location": false, - "result_estimate_count": 432000000.0, - "result_estimate_time": 0.35 - }, - "results": [ - { - "cite": null, - "cmpt_rank": 0, - "details": { - "heading": "Choose what you’re giving feedback on", - "img_url": null, - "text": "Choose what you’re giving feedback on|Stranger Things|2016 ‧ Horror ‧ 5 seasons|2016 ‧ Horror ‧ 5 seasons|About this result|Share|Share|Click to copy link|Click to copy link|Share link|Share link|Link copied|Link copied|Claim this knowledge panel|Send feedback|Overview|Overview|Overview|Cast|Cast|Cast|Episode guide|Episode guide|Episode guide|Watch show|Watch show|Watch show", - "type": "panel", - "urls": [ - { - "text": "Claim this knowledge panel", - "url": "https://posts.google.com/claim/?mid=/m/0131ln7y" - }, - { - "text": "Overview", - "url": "/search?sca_esv=a23961d46812ec7b&q=stranger+things&si=AL3DRZFVUWRMOUj1f9sMMa00uM4UfIAcEGDNoA6Yjutlv16vemeOI1grENxU5Nx6qS_L93wqG7-6AL6UePlOfZkgaS3Q6gOwpc6j02ypzhFLRqerVhUpG5M%3D&sa=X&ved=2ahUKEwijm4ivhsaSAxVERjABHXZfE2IQyNoBKAB6BAgREAA&ictx=1" - }, - { - "text": "Cast", - "url": "/search?sca_esv=a23961d46812ec7b&q=stranger+things+cast&si=AL3DRZHjIeJxefExFuR1fcGbKNKbrjp1oZM7BqykbiQP3Xq4Vo0oIcQ0wPDAcBPIP4WarXjyLrCQLFC-InbiX018f6EsDMtr1RBiH6ODm5zV70Ia8SR4X-8_9ESwZM85XinMCxg5tYipm_yhoTcTVqYsblXZ30mYOQ%3D%3D&sa=X&ved=2ahUKEwijm4ivhsaSAxVERjABHXZfE2IQyNoBKAB6BAgVEAA&ictx=1" - }, - { - "text": "Episode guide", - "url": "/search?sca_esv=a23961d46812ec7b&q=stranger+things+episodes&si=AL3DRZHREXfXpCqtX50QH0bNg7pp5I9CtRG6e8N9NjHhx69HoD8NIE2TJXEKAZsa6WcIXXtKuUUN7yaoRzcHggbfPsU7-PgMiHUCSC1E4yTY0fZ2fYzZchLboJwUCmIZvocpJMTcNVGD9k5YIkwjJ8eKvE75mSeKOw%3D%3D&sa=X&ved=2ahUKEwijm4ivhsaSAxVERjABHXZfE2IQyNoBKAB6BAgWEAA&ictx=1" - }, - { - "text": "Watch show", - "url": "/search?sca_esv=a23961d46812ec7b&q=stream+stranger+things&si=AL3DRZFGsU6JTPnqUgABt3oAaJqT5fQBvgtJZPq_3A6LGaCIPNwxrq4ANfJrUUmu4nVEDjaJknPefLcXl-_2HbbhyP7Me6UvvH_YSWFn6u89RJv2Xz8iVbBypEXd1R8aLxmjM6iRWgpc0KfcfkRjBPnqkzcOow7fXl3F6c0ZqXRh6cU-myggSOk%3D&sa=X&ved=2ahUKEwijm4ivhsaSAxVERjABHXZfE2IQyNoBKAB6BAgcEAA&ictx=1" - } - ] - }, - "error": null, - "section": "main", - "serp_rank": 0, - "sub_rank": 0, - "sub_type": "panel", - "text": null, - "title": "Stranger Things 2016 ‧ Horror ‧ 5 seasons About this result Share Share Facebook WhatsApp X Email Click to copy link Share link Link copied Claim this knowledge panel Send feedback Overview Cast Episode guide Watch show", - "type": "knowledge", - "url": null - }, - { - "cite": null, - "cmpt_rank": 1, - "details": { - "items": [ - { - "cost": "Subscription", - "text": "Netflix", - "url": "https://www.netflix.com/watch/80057281?source=35" - } - ], - "type": "hyperlinks" - }, - "error": null, - "section": "main", - "serp_rank": 1, - "sub_rank": 0, - "sub_type": null, - "text": null, - "title": "Watch show", - "type": "available_on", - "url": null - }, - { - "cite": "https://www.netflix.com › title", - "cmpt_rank": 2, - "details": null, - "error": null, - "section": "main", - "serp_rank": 2, - "sub_rank": 0, - "sub_type": null, - "text": "Jul 6, 2016 — When a young boy vanishes, a small town uncovers a mystery involving secret experiments, terrifying supernatural forces and one strange ...", - "title": "Watch Stranger Things | Netflix Official Site", - "type": "general", - "url": "https://www.netflix.com/title/80057281" - }, - { - "cite": null, - "cmpt_rank": 3, - "details": { - "items": [ - "Can I watch Stranger Things on anything other than Netflix?", - "It's stranger things eve Watch Stranger Things 5: Volume 2, out 26 ... Instagram https://www.instagram.com › reel Instagram https://www.instagram.com › reel", - "What streaming service is Stranger Things on?", - "Is Stranger Things season 5 release date on Netflix?" - ], - "type": "text" - }, - "error": null, - "section": "main", - "serp_rank": 3, - "sub_rank": 0, - "sub_type": null, - "text": "Can I watch Stranger Things on anything other than Netflix?<|>It's stranger things eve Watch Stranger Things 5: Volume 2, out 26 ... Instagram https://www.instagram.com › reel Instagram https://www.instagram.com › reel<|>What streaming service is Stranger Things on?<|>Is Stranger Things season 5 release date on Netflix?", - "title": null, - "type": "people_also_ask", - "url": null - }, - { - "cite": "9 comments · 4 weeks ago", - "cmpt_rank": 4, - "details": { - "items": [ - { - "text": "Need free streaming service where i could watch ...", - "url": "https://www.reddit.com/r/StrangerThingsMemes/comments/1p7bbqi/need_free_streaming_service_where_i_could_watch/" - }, - { - "text": "Where to watch Stranger Things all seasons for free ...", - "url": "https://www.reddit.com/r/InternetPiracy/comments/1dl80bg/where_to_watch_stranger_things_all_seasons_for/" - }, - { - "text": "More results from www.reddit.com", - "url": "/search?q=stream+stranger+things+site:www.reddit.com&sca_esv=a23961d46812ec7b&sa=X&ved=2ahUKEwijm4ivhsaSAxVERjABHXZfE2IQrQIoAnoECBAQAw" - } - ], - "type": "hyperlinks" - }, - "error": null, - "section": "main", - "serp_rank": 4, - "sub_rank": 0, - "sub_type": "submenu", - "text": "hello, where can I watch the new season of Stranger Things for free (if possible without ads) ? Anyone pls help? Read more", - "title": "hello, where can I watch the new season of Stranger ...", - "type": "general", - "url": "https://www.reddit.com/r/ask/comments/1q8fhjw/hello_where_can_i_watch_the_new_season_of/" - }, - { - "cite": "https://www.justwatch.com › tv-show › stranger-things", - "cmpt_rank": 5, - "details": null, - "error": null, - "section": "main", - "serp_rank": 5, - "sub_rank": 0, - "sub_type": null, - "text": "Currently you are able to watch \" Stranger Things\" streaming on Netflix , Netflix Standard with Ads. Besides streaming, the series is also available on Blu- ... Read more", - "title": "Stranger Things - streaming tv show online", - "type": "general", - "url": "https://www.justwatch.com/us/tv-show/stranger-things" - }, - { - "cite": "https://www.roku.com › What's On › TV Shows", - "cmpt_rank": 6, - "details": null, - "error": null, - "section": "main", - "serp_rank": 6, - "sub_rank": 0, - "sub_type": null, - "text": "Stranger Things , a teen tv shows series starring Winona Ryder, David Harbour, and Millie Brown is available to stream now. Watch it on Netflix on your Roku ... Read more", - "title": "How to watch and stream Stranger Things", - "type": "general", - "url": "https://www.roku.com/whats-on/tv-shows/stranger-things?id=1e3af5ec6fac591a97ebd6c2e209bd0c&srsltid=AfmBOorGCHGouXlr4G8gvHU0ebwGWqyeEShN65sjBzBJAAUzPM2GOHxQ" - }, - { - "cite": "https://www.tvguide.com › tvshows › stranger-things", - "cmpt_rank": 7, - "details": null, - "error": null, - "section": "main", - "serp_rank": 7, - "sub_rank": 0, - "sub_type": null, - "text": "Find out how to watch Stranger Things. Stream the latest seasons and episodes , watch trailers, and more for Stranger Things at TV Guide.", - "title": "Stranger Things - Where to Watch and Stream", - "type": "general", - "url": "https://www.tvguide.com/tvshows/stranger-things/1030459291/" - }, - { - "cite": "https://tv.apple.com › show › stranger-things", - "cmpt_rank": 8, - "details": null, - "error": null, - "section": "main", - "serp_rank": 8, - "sub_rank": 0, - "sub_type": null, - "text": "Mysteries unravel in a small Midwestern town in the 1980s , involving supernatural forces, secret experiments and one strange girl. Read more", - "title": "Stranger Things", - "type": "general", - "url": "https://tv.apple.com/us/show/stranger-things/umc.cmc.6a4s868u4ocy2a9zg6ochi2nd" - }, - { - "cite": "https://www.rottentomatoes.com › stranger_things", - "cmpt_rank": 9, - "details": null, - "error": null, - "section": "main", - "serp_rank": 9, - "sub_rank": 0, - "sub_type": null, - "text": "Watch Stranger Things — Season 5 with a subscription on Netflix . Stranger Things — Season 5. What to Know. Read more", - "title": "Season 5 – Stranger Things", - "type": "general", - "url": "https://www.rottentomatoes.com/tv/stranger_things/s05" - }, - { - "cite": "20+ comments · 2 years ago", - "cmpt_rank": 10, - "details": null, - "error": null, - "section": "main", - "serp_rank": 10, - "sub_rank": 0, - "sub_type": null, - "text": "Available on Prime, Tubi, and Roku Channel . I have a Roku tv and use the search feature to see what's available and where. “I don't want to ... Read more", - "title": "Where Can I Watch Stranger Things Apart From Netflix", - "type": "general", - "url": "https://www.facebook.com/groups/290086862161786/posts/1040629970440801/" - }, - { - "cite": "9.4M+ views · 3 months ago", - "cmpt_rank": 11, - "details": null, - "error": null, - "section": "main", - "serp_rank": 11, - "sub_rank": 0, - "sub_type": "video", - "text": "Props to Noah Schnapp for turning back into a child forStranger Thingsseason 5. Such dedication as an actor.", - "title": "Stranger Things 5 | First Five Minutes | Netflix", - "type": "general", - "url": "https://www.youtube.com/watch?v=vhFPHYgILN0" - }, - { - "cite": "https://www.imdb.com › title", - "cmpt_rank": 12, - "details": null, - "error": null, - "section": "main", - "serp_rank": 12, - "sub_rank": 0, - "sub_type": null, - "text": "A group of young friends witness supernatural forces and secret government exploits . As they search for answers, the children unravel a series of extraordinary ... Read more", - "title": "Stranger Things (TV Series 2016–2025)", - "type": "general", - "url": "https://www.imdb.com/title/tt4574334/" - }, - { - "cite": null, - "cmpt_rank": 13, - "details": { - "heading": "People also search for", - "items": [ - "Where can I watch Stranger Things Except Netflix", - "Stranger Things season 5", - "Where can I watch Stranger Things Season 1", - "Is Stranger Things free on Netflix", - "Stream Stranger Things Season 5", - "Stranger Things Season 2", - "Stranger Things videos", - "Stranger Things season 3" - ], - "type": "text" - }, - "error": null, - "section": "footer", - "serp_rank": 13, - "sub_rank": 0, - "sub_type": "people_also_search_for", - "text": "Where can I watch Stranger Things Except Netflix<|>Stranger Things season 5<|>Where can I watch Stranger Things Season 1<|>Is Stranger Things free on Netflix<|>Stream Stranger Things Season 5<|>Stranger Things Season 2<|>Stranger Things videos<|>Stranger Things season 3", - "title": null, - "type": "searches_related", - "url": null - } - ] -} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[ee36775407d7].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[ee36775407d7].json new file mode 100644 index 00000000..d7d8d89f --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[ee36775407d7].json @@ -0,0 +1,885 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 36600000.0, + "result_estimate_time": 0.41 + }, + "results": [ + { + "cite": "Amazon.com", + "cmpt_rank": 0, + "details": { + "n_reviews": "1.7K", + "price": "$20.88", + "rating": "4.5", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Planters Cocktail Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Weee!", + "cmpt_rank": 0, + "details": { + "price": "$6.79", + "source": "Weee!", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Red Skin Peanut 3 lbs.", + "type": "products", + "url": null + }, + { + "cite": "Aunt Ruby's Peanuts", + "cmpt_rank": 0, + "details": { + "price": "$15.95", + "source": "Aunt Ruby's Peanuts", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Roasted Redskin Peanuts - 40 oz. Tin.", + "type": "products", + "url": null + }, + { + "cite": "Hampton Farms", + "cmpt_rank": 0, + "details": { + "price": "$26.95", + "source": "Hampton Farms", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Extra Large Redskin Shelled Raw Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Walmart - Monroe Systems for Business", + "cmpt_rank": 0, + "details": { + "n_reviews": "9", + "price": "$29.99", + "rating": "4.0", + "source": "Walmart - Monroe Systems for Business", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Wakefield Virginia Peanuts Animal Grade Raw Wildlife Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 0, + "details": { + "price": "$21.49", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Virginia Diner Gourmet Natural Super Extra Large Red Skin Virginia Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Weee!", + "cmpt_rank": 0, + "details": { + "price": "$1.79", + "source": "Weee!", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Pinstar Supply Pinstar Premium Raw Red Skin Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Nuts.com", + "cmpt_rank": 0, + "details": { + "n_reviews": "53", + "price": "$4.99", + "rating": "4.9", + "source": "Nuts.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Nuts.com Raw Redskin Peanuts.", + "type": "products", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "items": [ + "Can you eat the red skin on peanuts?", + "Are raw red skin peanuts good for you?", + "Are Redskin peanuts healthier than regular peanuts?", + "What happened to red skin peanuts?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "Can you eat the red skin on peanuts?<|>Are raw red skin peanuts good for you?<|>Are Redskin peanuts healthier than regular peanuts?<|>What happened to red skin peanuts?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.feridies.com › prod_detail_list › Redskin-P...", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Redskin peanuts are the ones with the red skins still on them. This red skin is between the outer shell and the peanut and it is full of nutrients.", + "title": "Redskin Peanuts | Spanish Style Peanuts", + "type": "general", + "url": "https://www.feridies.com/prod_detail_list/Redskin-Peanuts" + }, + { + "cite": "https://www.vadiner.com › product › Redskin-Virginia-P...", + "cmpt_rank": 3, + "details": { + "price": "ating: 5", + "stock": " 30 reviews", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "These red skin peanuts are the best around and I have tried most of them. They are bigger, crunchier, and more tasty than others. Was this review helpful?", + "title": "Redskin Virginia Peanuts", + "type": "general", + "url": "https://www.vadiner.com/product/Redskin-Virginia-Peanuts" + }, + { + "cite": "https://www.whitleyspeanut.com › prod_detail_list › reds...", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "The first cousin to our famous \"Home Cooked\" Virginia Peanuts , they're shelled and salted, with skins on to bring out their unique flavor. You simply won't find ...", + "title": "Virginia Redskin Peanuts", + "type": "general", + "url": "https://www.whitleyspeanut.com/prod_detail_list/redskin-virginia-peanuts" + }, + { + "cite": "https://www.hubspeanuts.com › product › redskin-peanuts", + "cmpt_rank": 5, + "details": { + "price": "ating: 5", + "stock": " 5 reviews", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "Details. Underneath the outer shell, raw peanut kernels are enrobed in a beautiful red skin that is full of nutrients. Rather than removing this healthy little ...", + "title": "Redskin Peanuts", + "type": "general", + "url": "https://www.hubspeanuts.com/product/redskin-peanuts" + }, + { + "cite": "https://www.planters.com › product › planters-redskin-s...", + "cmpt_rank": 6, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": null, + "text": "These roasted peanuts are Kosher, grown in the USA and are packed in an airtight canister to ensure freshness. The convenient resealable lid makes it easy to ...", + "title": "PLANTERS® REDSKIN SPANISH PEANUTS, 12.5 OZ CAN", + "type": "general", + "url": "https://www.planters.com/product/planters-redskin-spanish-peanuts-12-5-oz-can/" + }, + { + "cite": "https://krema.com › products › hot-redskin-peanuts-16-...", + "cmpt_rank": 7, + "details": { + "price": "3.99", + "stock": " In stock", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "Hot Redskin Peanuts 16 oz. Bag. Redskin Peanuts are roasted in 100% Peanut Oil, then taken to the next level by adding a hot & Spicy Kick! Ingredients.", + "title": "Hot Redskin Peanuts 16 oz. Bag", + "type": "general", + "url": "https://krema.com/products/hot-redskin-peanuts-16-oz-bag" + }, + { + "cite": "www.reddit.com  ·  Jan 16, 2023", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": "IsItBullshit: eating the skins of peanuts is bad for you - Reddit", + "type": "discussions_and_forums", + "url": "https://www.reddit.com/r/IsItBullshit/comments/10d9bvz/isitbullshit_eating_the_skins_of_peanuts_is_bad/" + }, + { + "cite": "www.reddit.com  ·  May 8, 2020", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 1, + "sub_type": null, + "text": null, + "title": "Getting skin off red skin peanuts : r/AskCulinary - Reddit", + "type": "discussions_and_forums", + "url": "https://www.reddit.com/r/AskCulinary/comments/gfutjn/getting_skin_off_red_skin_peanuts/" + }, + { + "cite": "mybroadband.co.za  ·  Jan 20, 2023", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 2, + "sub_type": null, + "text": null, + "title": "So whatever happened to redskin peanuts? | MyBroadband Forum", + "type": "discussions_and_forums", + "url": "https://mybroadband.co.za/forum/threads/so-whatever-happened-to-redskin-peanuts.1221853/" + }, + { + "cite": "https://www.walmart.com › ... › Peanuts › Roasted Peanuts", + "cmpt_rank": 9, + "details": { + "price": "ating: 4.4", + "stock": " 222 reviews", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "PLANTERS Redskin Spanish Peanuts , Roasted Salted Peanuts , Plant Based Protein 12.5oz (1 Canister) ; Includes Added Sugars.00g ; Protein 7.00g ; Vitamin A, 0%.", + "title": "PLANTERS Redskin Spanish Peanuts, Roasted Salted ...", + "type": "general", + "url": "https://www.walmart.com/ip/PLANTERS-Redskin-Spanish-Peanuts-Roasted-Salted-Peanuts-Plant-Based-Protein-12-5oz-1-Canister/11964702" + }, + { + "cite": null, + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 0, + "sub_type": "shopping", + "text": "Shop deals", + "title": "Save with deals on apparel, electronics, and more", + "type": "promo", + "url": "/search?sca_esv=52b056e1a97f92b3&q=shop+deals&sa=X&ved=2ahUKEwjJ9YTMifCEAxVpOFkFHRlCC5IQ_hF6BAgzEAA" + }, + { + "cite": "https://www.amazon.com › Raw-Redskin-Peanuts-Unsal...", + "cmpt_rank": 11, + "details": { + "price": "ating: 4.6", + "stock": " 1,246 reviews", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "Guilt-Free Snacking - Indulge in the savory goodness of our peanut bags without worries. Each serving of these redskin Spanish peanuts only has 160 calories.", + "title": "Raw Red skin Peanuts, USA Grown, (Unsalted) 5LB Bag ...", + "type": "general", + "url": "https://www.amazon.com/Raw-Redskin-Peanuts-Unsalted-Bulk/dp/B0079EIIZ4" + }, + { + "cite": "Alive herbals", + "cmpt_rank": 12, + "details": { + "price": "$5.75", + "source": "Alive herbals", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Red Skin Raw Peanuts Buy at Low Cost.", + "type": "products", + "url": null + }, + { + "cite": "Oh! Nuts", + "cmpt_rank": 12, + "details": { + "price": "$4.99", + "source": "Oh! Nuts", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Dry Roasted Unsalted Redskin Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 12, + "details": { + "price": "$40.95", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Cream-Nut Red-skinned Virginia Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Walmart - Its Delish", + "cmpt_rank": 12, + "details": { + "n_reviews": "66", + "price": "$17.99", + "rating": "4.2", + "source": "Walmart - Its Delish", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Gourmet Raw Peanuts with Red Skin by Its Delish.", + "type": "products", + "url": null + }, + { + "cite": "Virginia Diner", + "cmpt_rank": 12, + "details": { + "price": "$20.95", + "source": "Virginia Diner", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Redskin Virginia Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Hope & Harmony Farms", + "cmpt_rank": 12, + "details": { + "price": "$24.95", + "source": "Hope & Harmony Farms", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Raw Virginia Redskin Peanuts 6 Lb. Bag.", + "type": "products", + "url": null + }, + { + "cite": "Nutstop.com", + "cmpt_rank": 12, + "details": { + "n_reviews": "3", + "price": "$2.69", + "rating": "4.0", + "source": "Nutstop.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Nutstop Red Skin Peanuts Roasted.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 12, + "details": { + "n_reviews": "17", + "price": "$21.99", + "rating": "4.2", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "We Got Nuts Raw Red skin Peanuts USA Grown.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 13, + "details": { + "price": "$47.95", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Hubs Peanuts Big Reds Premium Virginia Nuts with Red Skins Freshly Home Cooked Salted Flavor.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 13, + "details": { + "price": "$9.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Roasted and Salted Red Skin Spanish Peanuts, Cacahuate Snacks (Roasted & Salted Peanut, 1 Pound).", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 13, + "details": { + "n_reviews": "13", + "price": "$22.59", + "rating": "4.2", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "The Peanut Shop of Williamsburg Handcooked Virginia Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "hillsonnut.com", + "cmpt_rank": 13, + "details": { + "price": "$3.69", + "source": "hillsonnut.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Peanuts: Redskin.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 13, + "details": { + "price": "$34.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 33, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Red-Skin Peanuts Shelled Roasted Unsalted Jumbo Runner Bulk 10 Pound Box.", + "type": "products", + "url": null + }, + { + "cite": "Hope & Harmony Farms", + "cmpt_rank": 13, + "details": { + "price": "$30.95", + "source": "Hope & Harmony Farms", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 34, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Salted Virginia Red Skin Peanuts 40 Oz..", + "type": "products", + "url": null + }, + { + "cite": "sayweee.com", + "cmpt_rank": 13, + "details": { + "price": "$6.79", + "source": "sayweee.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 35, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Healthy Family Red Skin Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Belmont Peanuts", + "cmpt_rank": 13, + "details": { + "price": "$16.95", + "source": "Belmont Peanuts", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 36, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Redskin Peanuts | Belmont Peanuts | 20 oz | Single Can.", + "type": "products", + "url": null + }, + { + "cite": "https://hopeandharmonyfarms.com › SHOP ALL", + "cmpt_rank": 14, + "details": { + "price": "ating: 5", + "stock": " 2 reviews", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 37, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "SALTED VIRGINIA RED SKIN PEANUTS (3) 10 OZ. $34.95. Super extra-large Virginia peanuts, hand cooked with the skins on, in 100.", + "title": "salted virginia red skin peanuts 40 oz.", + "type": "general", + "url": "https://hopeandharmonyfarms.com/salted-virginia-red-skin-peanuts-40-oz/" + }, + { + "cite": "Whitley's Peanut Factory", + "cmpt_rank": 15, + "details": { + "price": "$19.00", + "source": "Whitley's Peanut Factory", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 38, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "16 oz. Pouch Bag Salted Redskin Virginia Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "Nutstop.com", + "cmpt_rank": 15, + "details": { + "price": "$2.69", + "source": "Nutstop.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 39, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Raw Redskin Peanuts - 1lb Bag - Bulk Sizes Available.", + "type": "products", + "url": null + }, + { + "cite": "We Got Nuts", + "cmpt_rank": 15, + "details": { + "n_reviews": "62", + "price": "$19.95", + "rating": "4.8", + "source": "We Got Nuts", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 40, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Raw Redskin Peanuts Premium Nuts From We Got Nuts.", + "type": "products", + "url": null + }, + { + "cite": "Po Wing Online", + "cmpt_rank": 15, + "details": { + "price": "$2.99", + "source": "Po Wing Online", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 41, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Heng Cheong Loong Red Skin Peanuts.", + "type": "products", + "url": null + }, + { + "cite": "theloadedkitchen.com", + "cmpt_rank": 15, + "details": { + "price": "$2.19", + "source": "theloadedkitchen.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 42, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Hampton Farms Raw Peanuts Redskin.", + "type": "products", + "url": null + }, + { + "cite": "Walmart - Monroe Systems for Business", + "cmpt_rank": 15, + "details": { + "n_reviews": "3", + "price": "$109.99", + "rating": "4.3", + "source": "Walmart - Monroe Systems for Business", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 43, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Virginia Peanuts Animal Grade Raw Red Skin Peanuts for Squirrels Birds Deer Pigs and A Wide Variety of Wildlife/Bulk Nuts/Blu.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 15, + "details": { + "price": "$14.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 44, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Raw Red Skin Peanuts (Uncooked, Unsalted) 3 LB. (48oz), Product of USA.", + "type": "products", + "url": null + }, + { + "cite": "Feridies", + "cmpt_rank": 15, + "details": { + "price": "$83.98", + "source": "Feridies", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 45, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "36oz Redskin Virginia Peanuts - 4 Pack.", + "type": "products", + "url": null + }, + { + "cite": null, + "cmpt_rank": 16, + "details": { + "heading": "Related searches", + "items": [ + "Red skin peanuts vs regular peanuts", + "Red skin peanuts nutrition", + "red skin peanuts benefits", + "red skin peanuts raw", + "red skin peanuts protein", + "Where to buy red skin peanuts" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 46, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "Red skin peanuts vs regular peanuts<|>Red skin peanuts nutrition<|>red skin peanuts benefits<|>red skin peanuts raw<|>red skin peanuts protein<|>Where to buy red skin peanuts", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 17, + "details": { + "items": [ + "Health benefits", + "Food Nutrition", + "Growing area", + "Taste" + ], + "type": "panel" + }, + "error": null, + "section": "rhs", + "serp_rank": 47, + "sub_rank": 0, + "sub_type": "panel_rhs", + "text": null, + "title": "Things to know", + "type": "knowledge", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[eef22ed8a7ee].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[eef22ed8a7ee].json new file mode 100644 index 00000000..b2dcb109 --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[eef22ed8a7ee].json @@ -0,0 +1,342 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard-overview", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 846000.0, + "result_estimate_time": 0.96 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "heading": "I Believe That We Will Win (World Anthem)", + "img_url": null, + "text": "I Believe That We Will Win (World Anthem)|Song by Pitbull|Song by Pitbull|Overview|Overview|Overview|Lyrics|Lyrics|Lyrics|Videos|Videos|Videos|Listen|Listen|Listen", + "type": "panel" + }, + "error": null, + "section": "main", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": "panel", + "text": null, + "title": "I Believe That We Will Win (World Anthem) Song by Pitbull Overview Lyrics Videos Listen", + "type": "knowledge", + "url": null + }, + { + "cite": null, + "cmpt_rank": 1, + "details": { + "heading": null, + "img_url": null, + "type": "panel", + "urls": [ + { + "text": "2:57|YouTube| • |Pitbull|Pitbull - I Believe That We Will Win | World Anthem (Official ...", + "url": "https://www.youtube.com/watch?v=ZxMaHi91RkE" + }, + { + "text": "2:28|YouTube| • |Pitbull|Pitbull - I Believe That We Will Win [World Anthem] (Official ...|Pitbull - I Believe That We Will Win [World Anthem] (Official Lyric Video) Listen Here: https://mr305.ffm.to/believeanthem Subscribe: ...|Apr 13, 2020", + "url": "https://www.youtube.com/watch?v=BBre0xNcr5c" + }, + { + "text": "Artist|Pitbull", + "url": "/search?sca_esv=52b056e1a97f92b3&q=Armando+Christian+Perez&si=AKbGX_qWtsfHufXsq_1jeDkJp50FstNngDxsch3EVTUjn7imcCmqWbQaFmXWpbSWMcBqD-oLsw1Jh67pG418bH1ruu8l4eB8ObKQ4pmoSKEhF0l_VPB4uBXJjZAgODyBApYfRmtjhZFzIZISJAFZupuc0O0a61xdKki1nXqccloaXyQMfkb8dNHzfC0qNRNTUxG1tfQ29NmtLcghZ695thQTX1cinAgS2A%3D%3D&sa=X&ved=2ahUKEwjszMrshPCEAxUhOWIAHeK3BTQQ18AJegQIFhAB" + }, + { + "text": "Released|2020", + "url": "/search?sca_esv=52b056e1a97f92b3&q=pitbull+i+believe+that+we+will+win+(world+anthem)+released&stick=H4sIAAAAAAAAAC3MOw7CMBAFwAqJBgpOsGWgsdxQ-DLIn0e8YuNE9jqWOA4nhYJ2ijmeLyczG2ufmsP2tvfb9IrOLL1xNBVxrYnL_IjSm6K6CoFvoOQVn4PbWEMXIaYAYewgzV5pgAb_eHChaaxVEvmiGcuV_kH6Ai2GCcV4AAAA&sa=X&ved=2ahUKEwjszMrshPCEAxUhOWIAHeK3BTQQ18AJegQIFRAB" + }, + { + "text": "", + "url": "https://www.cnn.com/2020/04/13/entertainment/pitbull-new-song-i-believe-that-we-will-win-trnd/index.html" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "featured_results", + "text": "2:57 YouTube • Pitbull Pitbull - I Believe That We Will Win | World Anthem (Official ... 2:28 YouTube • Pitbull Pitbull - I Believe That We Will Win [World Anthem] (Official ... Pitbull - I Believe That We Will Win [World Anthem] (Official Lyric Video) Listen Here: https://mr305.ffm.to/believeanthem Subscribe: ... Apr 13, 2020 Artist Pitbull Released 2020 CNN Pitbull releases uplifting song 'I Believe That We Will Win ... Pitbull is bringing the world together through his music. “Mr. Worldwide” really is living up to his nickname with his latest single, ... Apr 13, 2020", + "title": null, + "type": "knowledge", + "url": "https://www.youtube.com/watch?v=ZxMaHi91RkE" + }, + { + "cite": null, + "cmpt_rank": 2, + "details": { + "heading": "Lyrics", + "img_url": null, + "type": "panel", + "urls": [ + { + "text": "Musixmatch", + "url": "https://www.musixmatch.com/" + } + ] + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 0, + "sub_type": "lyrics", + "text": null, + "title": "Lyrics", + "type": "knowledge", + "url": null + }, + { + "cite": "YouTube", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 0, + "sub_type": "unspecified-3", + "text": null, + "title": "Pitbull - I Believe That We Will Win [World Anthem] (Official ... YouTube · Pitbull 2 minutes, 28 seconds Apr 12, 2020", + "type": "videos", + "url": "https://www.youtube.com/watch?v=BBre0xNcr5c" + }, + { + "cite": "YouTube", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 1, + "sub_type": "unspecified-3", + "text": null, + "title": "Pitbull - I Believe That We Will Win [World Anthem] (Audio Video) YouTube · Pitbull 2 minutes, 28 seconds Apr 24, 2020", + "type": "videos", + "url": "https://www.youtube.com/watch?v=A9p50gpCf-k" + }, + { + "cite": "YouTube", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 2, + "sub_type": "unspecified-3", + "text": null, + "title": "Pitbull - I Believe That We Will Win | World Anthem - Thombs ... YouTube · Pitbull 3 minutes Jun 26, 2020", + "type": "videos", + "url": "https://www.youtube.com/watch?v=vzGbIecs4_o" + }, + { + "cite": "YouTube", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 3, + "sub_type": "unspecified-3", + "text": null, + "title": "Pitbull - I Believe That We Will Win | World Anthem (Official BTS) YouTube · Pitbull 1 minute, 50 seconds May 21, 2020", + "type": "videos", + "url": "https://www.youtube.com/watch?v=BdEP6pDZWFE" + }, + { + "cite": null, + "cmpt_rank": 4, + "details": { + "items": [ + "Who sings I believe we will win?", + "What song is the anthem by Pitbull based on?", + "What song made Pitbull famous?", + "Who has Pitbull sang with?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 0, + "sub_type": null, + "text": "Who sings I believe we will win?<|>What song is the anthem by Pitbull based on?<|>What song made Pitbull famous?<|>Who has Pitbull sang with?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.cnn.com › 2020/04/13 › entertainment › pi...", + "cmpt_rank": 5, + "details": null, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 0, + "sub_type": null, + "text": "Apr 13, 2020 — Worldwide” really is living up to his nickname with his latest single, “ I Believe That We Will Win .” The song is an anthem about the current ...", + "title": "Pitbull releases uplifting song 'I Believe That We Will Win' ...", + "type": "general", + "url": "https://www.cnn.com/2020/04/13/entertainment/pitbull-new-song-i-believe-that-we-will-win-trnd/index.html" + }, + { + "cite": "36.6K+ views · 3 years ago", + "cmpt_rank": 6, + "details": { + "duration": "2:38", + "source": null, + "type": "video" + }, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": "video", + "text": "Pitbull-I Believe That We Will Win[World Anthem] (Lyric Video) Subscribe: http://bitly.com/Pibull_YouTube Follow Pitbull: Instagram: ...", + "title": "Pitbull - I Believe That We Will Win (Lyrics) - YouTube", + "type": "general", + "url": "https://www.youtube.com/watch?v=Y1ZsKjCYCbs" + }, + { + "cite": "https://open.spotify.com › track", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": null, + "text": "I Believe That We Will Win ( World Anthem )20202:27. Sign in to see lyrics and listen to the full track. Sign up. Pitbull . Artist. Pitbull  ...", + "title": "I Believe That We Will Win (World Anthem)", + "type": "general", + "url": "https://open.spotify.com/track/4Sq71vakNYzewI1nnZItEC" + }, + { + "cite": "https://www.billboard.com › Music › Latin", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 0, + "sub_type": null, + "text": "Apr 13, 2020 — His new song “ I Believe That We Will Win ” is no exception, in fact, “that message is needed right now more than ever,” he tells Billboard. It ...", + "title": "Pitbull Drops Uplifting Song 'I Believe That We Will Win'", + "type": "general", + "url": "https://www.billboard.com/music/latin/pitbull-i-believe-that-we-will-win-listen-9356237/" + }, + { + "cite": "753.2K+ views · 3 years ago", + "cmpt_rank": 9, + "details": { + "duration": "2:28", + "source": null, + "type": "video" + }, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 0, + "sub_type": "video", + "text": "Provided to YouTube by Universal Music Group I Believe That We Will Win (World Anthem) ·Pitbull I Believe That We Will Win(World Anthem) ...", + "title": "I Believe That We Will Win (World Anthem) - YouTube", + "type": "general", + "url": "https://www.youtube.com/watch?v=qLxpmLl4fto" + }, + { + "cite": "83.7K+ views · 3 years ago", + "cmpt_rank": 10, + "details": { + "duration": "3:04", + "source": null, + "type": "video" + }, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": "video", + "text": "WEbelieve that we will win! Mr. Worldwide brings the heat with this brand new release, and we're obsessed with the song's globally positive ...", + "title": "“I Believe That We Will Win (World Anthem)” || Pitbull - YouTube", + "type": "general", + "url": "https://www.youtube.com/watch?v=nHi2NPv2Bz0" + }, + { + "cite": "https://www.mtv.com › news › pitbull-i-believe-that-we-...", + "cmpt_rank": 11, + "details": null, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": null, + "text": "May 7, 2020 — Pitbull's hopeful new ' I Believe That We Will Win ( World Anthem )' video finds Mr. Worldwide summoning the power of global community to keep ...", + "title": "Pitbull's 'I Believe That We Will Win' Shows Global Unity", + "type": "general", + "url": "https://www.mtv.com/news/w4i14l/pitbull-i-believe-that-we-will-win-world-anthem-video" + }, + { + "cite": "https://www.amazon.com › Believe-That-Will-World-A...", + "cmpt_rank": 12, + "details": null, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 0, + "sub_type": null, + "text": "Check out I Believe That We Will Win ( World Anthem ) by Pitbull on Amazon Music. Stream ad-free or purchase CD's and MP3s now on Amazon.com.", + "title": "Pitbull - I Believe That We Will Win (World Anthem)", + "type": "general", + "url": "https://www.amazon.com/Believe-That-Will-World-Anthem/dp/B086R69G96" + }, + { + "cite": null, + "cmpt_rank": 13, + "details": { + "heading": "Related searches", + "items": [ + "pitbull i believe that we will win meaning", + "i believe that we will win football chant", + "we will win song", + "i believe that we will win soccer", + "i believe that we will win'' chant origin", + "i believe that we will win remix", + "i believe we will win gif", + "I believe that we will win Clean" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "pitbull i believe that we will win meaning<|>i believe that we will win football chant<|>we will win song<|>i believe that we will win soccer<|>i believe that we will win'' chant origin<|>i believe that we will win remix<|>i believe we will win gif<|>I believe that we will win Clean", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 14, + "details": null, + "error": "no subcomponents parsed", + "section": "rhs", + "serp_rank": 17, + "sub_rank": 0, + "sub_type": null, + "text": "Complementary Results<|>Listen<|>YouTube Music<|>Apple Music<|>Spotify<|>iHeartRadio<|>About<|>Artist<|>:<|>Pitbull<|>Released<|>:<|>2020<|>Album<|>:<|>I Believe That We Will Win (World Anthem)<|>Genre<|>:<|>Pop<|>Feedback<|>People also search for<|>We Will Rock You<|>Queen<|>3 to Tango<|>Pitbull<|>Eye of the Tiger<|>Survivor<|>U Can’t Touch This<|>MC Hammer<|>See more", + "title": null, + "type": "knowledge_rhs", + "url": null + } + ] +} diff --git a/tests/__snapshots__/test_parse_serp/test_parse_serp[f10d116b26c6].json b/tests/__snapshots__/test_parse_serp/test_parse_serp[f10d116b26c6].json new file mode 100644 index 00000000..7b6602ed --- /dev/null +++ b/tests/__snapshots__/test_parse_serp/test_parse_serp[f10d116b26c6].json @@ -0,0 +1,951 @@ +{ + "features": { + "captcha": false, + "infinity_scroll": true, + "language": "en", + "main_layout": "standard", + "notice_no_results": false, + "notice_server_error": false, + "notice_shortened_query": false, + "overlay_precise_location": false, + "result_estimate_count": 1280000000.0, + "result_estimate_time": 0.53 + }, + "results": [ + { + "cite": null, + "cmpt_rank": 0, + "details": { + "items": [ + { + "text": "TOPS Products", + "url": "/search?q=tops+products+file+folder&shoprs=CAEYASoLZmlsZSBmb2xkZXIyGwgBEg1UT1BTIFByb2R1Y3RzOggIg_M8EIy6QFisyyBgAQ&sa=X&ved=2ahUKEwiBlKKNjfCEAxXmD1kFHSlnCzoQip4GKAB6BAgNEAM" + }, + { + "text": "Get it fast", + "url": "/search?q=file+folder&shoprs=CAESFpoBEwoRCAMqCQoDVVNEEMCEPTgBSAEYFCoLZmlsZSBmb2xkZXIyLQgUEgtHZXQgaXQgZmFzdCIWmgETChEIAyoJCgNVU0QQwIQ9OAFIASoEEAEYAWAB&sa=X&ved=2ahUKEwiBlKKNjfCEAxXmD1kFHSlnCzoQip4GKAF6BAgNEAQ" + }, + { + "text": "Nearby", + "url": "/search?q=file+folder+nearby&shoprs=CAEYAyoLZmlsZSBmb2xkZXIyDAgDEgZOZWFyYnkYAlisyyBgAQ&sa=X&ved=2ahUKEwiBlKKNjfCEAxXmD1kFHSlnCzoQip4GKAJ6BAgNEAU" + }, + { + "text": "Smead", + "url": "/search?q=smead+file+folder&shoprs=CAEYASoLZmlsZSBmb2xkZXIyEwgBEgVTbWVhZDoICIPzPBCL2FBYrMsgYAE&sa=X&ved=2ahUKEwiBlKKNjfCEAxXmD1kFHSlnCzoQip4GKAN6BAgNEAY" + }, + { + "text": "Used", + "url": "/search?q=used+file+folder&shoprs=CAESBTIDCgEDGAoqC2ZpbGUgZm9sZGVyMhMIChIEVXNlZCIFMgMKAQMqAhgBWKzLIGAB&sa=X&ved=2ahUKEwiBlKKNjfCEAxXmD1kFHSlnCzoQip4GKAR6BAgNEAc" + }, + { + "text": "Under $5", + "url": "/search?q=file+folder+under+$5&shoprs=CAESDRILEQAAAADQElNBGAEYBSoLZmlsZSBmb2xkZXIyIQgFEghVbmRlciAkNRgCIg0SCxEAAAAA0BJTQRgBKgIYAVisyyBgAQ&sa=X&ved=2ahUKEwiBlKKNjfCEAxXmD1kFHSlnCzoQip4GKAV6BAgNEAg" + }, + { + "text": "3 Ring", + "url": "/search?q=3+ring+file+folder&shoprs=CAEYASoLZmlsZSBmb2xkZXIyFAgBEgYzIFJpbmc6CAiugDsQr4A7WKzLIGAB&sa=X&ved=2ahUKEwiBlKKNjfCEAxXmD1kFHSlnCzoQip4GKAZ6BAgNEAk" + }, + { + "text": "Small business", + "url": "/search?q=file+folder&shoprs=CAESAmoAGBYqC2ZpbGUgZm9sZGVyMhwIFhIOU21hbGwgYnVzaW5lc3MiAmoAKgQQARgBYAE&sa=X&ved=2ahUKEwiBlKKNjfCEAxXmD1kFHSlnCzoQip4GKAd6BAgNEAo" + } + ], + "type": "hyperlinks" + }, + "error": null, + "section": "header", + "serp_rank": 0, + "sub_rank": 0, + "sub_type": null, + "text": null, + "title": null, + "type": "top_image_carousel", + "url": null + }, + { + "cite": "Office Depot", + "cmpt_rank": 1, + "details": { + "n_reviews": "8.6K", + "price": "$18.49", + "rating": "4.8", + "source": "Office Depot", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 1, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Office Depot File Folders 1/3 Cut.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 1, + "details": { + "n_reviews": "347", + "price": "$14.30", + "rating": "4.3", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 2, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Basics Hanging Folders Letter.", + "type": "products", + "url": null + }, + { + "cite": "Staples", + "cmpt_rank": 1, + "details": { + "n_reviews": "209", + "price": "$12.29", + "rating": "4.7", + "source": "Staples", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 3, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Staples File Folder 1/3 Cut Tab.", + "type": "products", + "url": null + }, + { + "cite": "Target", + "cmpt_rank": 1, + "details": { + "n_reviews": "53", + "price": "$3.99", + "rating": "4.8", + "source": "Target", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 4, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "up & up Manila File Folders.", + "type": "products", + "url": null + }, + { + "cite": "Walmart - BOP CO", + "cmpt_rank": 1, + "details": { + "price": "$14.98", + "source": "Walmart - BOP CO", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 5, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Better Office Products Lined File Folders 24 Pack Notes Tabbed File Folders.", + "type": "products", + "url": null + }, + { + "cite": "Office Depot", + "cmpt_rank": 1, + "details": { + "n_reviews": "374", + "price": "$25.99", + "rating": "4.6", + "source": "Office Depot", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 6, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Pendaflex Expanding File.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 1, + "details": { + "n_reviews": "43", + "price": "$9.49", + "rating": "4.7", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 7, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Amazon Basics Expanding Organizer File Folder.", + "type": "products", + "url": null + }, + { + "cite": "Smead Manufacturing", + "cmpt_rank": 1, + "details": { + "n_reviews": "2.5K", + "price": "$25.96", + "rating": "4.7", + "source": "Smead Manufacturing", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 8, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Smead 1/3-Cut Tab File Folders.", + "type": "products", + "url": null + }, + { + "cite": "https://www.amazon.com › File-Folders", + "cmpt_rank": 2, + "details": null, + "error": null, + "section": "main", + "serp_rank": 9, + "sub_rank": 0, + "sub_type": null, + "text": "Pendaflex File Folders , Letter Size, 8-1/2\" x 11\", Classic Manila, 1/3-Cut Tabs in Left, Right, Center Positions, 100 Per Box (65213).", + "title": "File Folders", + "type": "general", + "url": "https://www.amazon.com/File-Folders/b?ie=UTF8&node=1069608" + }, + { + "cite": "Amazon.com", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 10, + "sub_rank": 0, + "sub_type": "medium", + "text": null, + "title": "Pendaflex File Folders, Letter Size, 8-1/2\" x 11\", Classic Manila, 1/3-Cut Tabs in Left, Right, Center Positions, 100 Per Box (65213) Amazon.com", + "type": "images", + "url": "https://www.amazon.com/Pendaflex-Folders-Classic-Positions-65213/dp/B0722L14L3" + }, + { + "cite": "Amazon", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 11, + "sub_rank": 1, + "sub_type": "medium", + "text": null, + "title": "Smead Reversible Tab File Folders, 1/2-Cut Tab (15394) Amazon", + "type": "images", + "url": "https://www.amazon.com/Smead-Reversible-Folder-Assorted-15394/dp/B00FSAO4UO" + }, + { + "cite": "Smead", + "cmpt_rank": 3, + "details": null, + "error": null, + "section": "main", + "serp_rank": 12, + "sub_rank": 2, + "sub_type": "medium", + "text": null, + "title": "Smead Organized Up Heavyweight Vertical File Folders, Assorted Bright Tones - 6 pack Smead", + "type": "images", + "url": "https://www.smead.com/products/heavyweight-vertical-file-folder" + }, + { + "cite": "https://www.target.com › ... › Filing › Files", + "cmpt_rank": 4, + "details": null, + "error": null, + "section": "main", + "serp_rank": 13, + "sub_rank": 0, + "sub_type": null, + "text": "Shop Target for File Folders you will love at great low prices. Free shipping on orders of $35+ or same-day pick-up in store.", + "title": "File Folders", + "type": "general", + "url": "https://www.target.com/c/file-folders-files-filing-school-office-supplies/-/N-4smja" + }, + { + "cite": "Target", + "cmpt_rank": 5, + "details": { + "n_reviews": "165", + "price": "$5.39", + "rating": "4.6", + "source": "Target", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 14, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "up & up Hanging File Folders Letter Size Green.", + "type": "products", + "url": null + }, + { + "cite": "kingstationary.com", + "cmpt_rank": 5, + "details": { + "n_reviews": "1", + "price": "$28.99", + "rating": "5.0", + "source": "kingstationary.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 15, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Amazon Basics AMZ401 File Folders.", + "type": "products", + "url": null + }, + { + "cite": "Office Depot", + "cmpt_rank": 5, + "details": { + "n_reviews": "3.6K", + "price": "$27.99", + "rating": "4.7", + "source": "Office Depot", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 16, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Office Depot Brand Hanging Folders Cut.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 5, + "details": { + "n_reviews": "5", + "price": "$12.82", + "rating": "4.4", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 17, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "Basics File Folders Letter.", + "type": "products", + "url": null + }, + { + "cite": "Walmart", + "cmpt_rank": 5, + "details": { + "n_reviews": "59", + "price": "$10.99", + "rating": "4.8", + "source": "Walmart", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 18, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "C Line Write On Poly File Jackets.", + "type": "products", + "url": null + }, + { + "cite": "Temu", + "cmpt_rank": 5, + "details": { + "n_reviews": "131", + "price": "$2.98", + "rating": "4.6", + "source": "Temu", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 19, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Skydue Letter A4 Paper Expanding File Folder Pockets Accordion Document Organizer. Low price.", + "type": "products", + "url": null + }, + { + "cite": "eBay - tzurie-72", + "cmpt_rank": 5, + "details": { + "n_reviews": "2.5K", + "price": "$31.33", + "rating": "4.7", + "source": "eBay - tzurie-72", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 20, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Pendaflex File Folders.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 5, + "details": { + "n_reviews": "15", + "price": "$6.95", + "rating": "4.9", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 21, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Dunwell Colored Plastic Folders Folders.", + "type": "products", + "url": null + }, + { + "cite": null, + "cmpt_rank": 6, + "details": { + "items": [ + "What is a file and a folder?", + "How do I find my folder?", + "Where do I make a file folder?", + "What is a folder icon?" + ], + "type": "text" + }, + "error": null, + "section": "main", + "serp_rank": 22, + "sub_rank": 0, + "sub_type": null, + "text": "What is a file and a folder?<|>How do I find my folder?<|>Where do I make a file folder?<|>What is a folder icon?", + "title": null, + "type": "people_also_ask", + "url": null + }, + { + "cite": "https://www.officedepot.com › Office Supplies", + "cmpt_rank": 7, + "details": null, + "error": null, + "section": "main", + "serp_rank": 23, + "sub_rank": 0, + "sub_type": null, + "text": "There are six main types of folders: file folders , hanging file folders , expanding file folders , classification folders, zip folders, and pocket folders. Manila ...", + "title": "Filing And Folders", + "type": "general", + "url": "https://www.officedepot.com/b/filing-and-folders/N-3469" + }, + { + "cite": "https://www.walmart.com › ... › Folders › Filing Products", + "cmpt_rank": 8, + "details": null, + "error": null, + "section": "main", + "serp_rank": 24, + "sub_rank": 0, + "sub_type": null, + "text": "Shop for Filing Products in Folders. Buy products such as Pendaflex Hanging File Folders , Standard Green, Letter Size, 50 Per Box at Walmart and save.", + "title": "Filing Products in Folders(1000+)", + "type": "general", + "url": "https://www.walmart.com/browse/office-supplies/filing-products/1229749_4464616_9115471" + }, + { + "cite": "https://www.smead.com › collections › top-tab-file-fold...", + "cmpt_rank": 9, + "details": { + "price": "ree 6–7 day delivery over $50", + "stock": " 30", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 25, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "Assorted Colors Poly File Folders , Straight-Cut Tab, Letter. Regular price $6.95. Regular price $6.95 Sale price $6.95. Heavyweight Vertical File Folder , ...", + "title": "‏‏‎ ‎Top Tab File Folders", + "type": "general", + "url": "https://www.smead.com/collections/top-tab-file-folders" + }, + { + "cite": "https://www.staples.com › ... › Filing Folders", + "cmpt_rank": 10, + "details": null, + "error": null, + "section": "main", + "serp_rank": 26, + "sub_rank": 0, + "sub_type": null, + "text": "Staples carries a variety of file folders including standard manila folders, colorful filing folders , and letter and legal-sized options.", + "title": "File Folders & Manila Folders", + "type": "general", + "url": "https://www.staples.com/file-folders/cat_CL141669" + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 11, + "details": { + "n_reviews": "2", + "price": "$15.99", + "rating": "4.5", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 27, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Fanwu Plastic File Folder Poly Envelope Expanding File Wallet Document Folder with Snap Button Closure.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 11, + "details": { + "n_reviews": "23", + "price": "$9.00", + "rating": "4.8", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 28, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "Avery Expanding File.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 11, + "details": { + "price": "$7.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 29, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "HABGP 9 Pcs Manilla Folders 8.5 x 11 Inch 1/3-Cut Tab Colored File Folders Letter.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 11, + "details": { + "n_reviews": "1", + "price": "$15.99", + "rating": "4.0", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 30, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "ThinkTex Accordian File Organizer 12 Pockets Expanding File Folders.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 11, + "details": { + "price": "$9.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 31, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Sooez Expanding File Folder Organizer with Sticky Labels 13 File Organizer Accordion File Organizer.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 11, + "details": { + "n_reviews": "2K", + "price": "$13.20", + "rating": "4.7", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 32, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Smead SuperTab File Folders 1/3 Cut.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 11, + "details": { + "n_reviews": "10", + "price": "$13.99", + "rating": "5.0", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 33, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "EOOUT 18 Pack Decorative File Folders Floral Folders Cute File Folder Letter Size Colored File Folders,1/3-Cut Tabs.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com", + "cmpt_rank": 11, + "details": { + "n_reviews": "235", + "price": "$27.75", + "rating": "4.7", + "source": "Amazon.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 34, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Pendaflex Manila Fastener Folders 1/3-Cut Tabs.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 12, + "details": { + "price": "$11.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 35, + "sub_rank": 0, + "sub_type": "grid", + "text": null, + "title": "Mr. Pen File Folders 1/3 Cut Tab.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 12, + "details": { + "price": "$16.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 36, + "sub_rank": 1, + "sub_type": "grid", + "text": null, + "title": "ThinkTex Plastic Expanding File Folder Transparent envelopes with Snap Closure.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 12, + "details": { + "price": "$14.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 37, + "sub_rank": 2, + "sub_type": "grid", + "text": null, + "title": "Sooez 24 Pockets Expanding File Folder with Blank Labels, According File Organizer with Expandable Cover, Desktop Accordion Folders Letter A4 Paper.", + "type": "products", + "url": null + }, + { + "cite": "Target", + "cmpt_rank": 12, + "details": { + "n_reviews": "7", + "price": "$6.99", + "rating": "4.8", + "source": "Target", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 38, + "sub_rank": 3, + "sub_type": "grid", + "text": null, + "title": "U Brands File Folders Subtle Textures.", + "type": "products", + "url": null + }, + { + "cite": "Walmart", + "cmpt_rank": 12, + "details": { + "n_reviews": "363", + "price": "$3.37", + "rating": "4.5", + "source": "Walmart", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 39, + "sub_rank": 4, + "sub_type": "grid", + "text": null, + "title": "Pen+Gear 3-Tab Manila File Folders, 25 Count, Letter Size.", + "type": "products", + "url": null + }, + { + "cite": "Staples", + "cmpt_rank": 12, + "details": { + "n_reviews": "3.1K", + "price": "$15.39", + "rating": "4.4", + "source": "Staples", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 40, + "sub_rank": 5, + "sub_type": "grid", + "text": null, + "title": "Staples Hanging File Folder 5-Tab Letter Size Standard.", + "type": "products", + "url": null + }, + { + "cite": "tops-products.com", + "cmpt_rank": 12, + "details": { + "n_reviews": "399", + "price": "$20.73", + "rating": "4.5", + "source": "tops-products.com", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 41, + "sub_rank": 6, + "sub_type": "grid", + "text": null, + "title": "Pendaflex Divide It Up File Folder.", + "type": "products", + "url": null + }, + { + "cite": "Amazon.com - Seller", + "cmpt_rank": 12, + "details": { + "price": "$9.99", + "source": "Amazon.com - Seller", + "type": "ratings" + }, + "error": null, + "section": "main", + "serp_rank": 42, + "sub_rank": 7, + "sub_type": "grid", + "text": null, + "title": "Maimis Expanding File Folder 24 Pocket Accordion A4 Letter Size Portable Document Organizer with Colored Tabs. SALE.", + "type": "products", + "url": null + }, + { + "cite": null, + "cmpt_rank": 13, + "details": null, + "error": null, + "section": "main", + "serp_rank": 43, + "sub_rank": 0, + "sub_type": "shopping", + "text": "Shop deals", + "title": "Save with deals on apparel, electronics, and more", + "type": "promo", + "url": "/search?sca_esv=52b056e1a97f92b3&q=shop+deals&sa=X&ved=2ahUKEwiBlKKNjfCEAxXmD1kFHSlnCzoQ_hF6BAgjEAA" + }, + { + "cite": "https://www.levenger.com › collections › filing-folders", + "cmpt_rank": 14, + "details": null, + "error": null, + "section": "main", + "serp_rank": 44, + "sub_rank": 0, + "sub_type": null, + "text": "Create a filing system that works for you. Discover the durability of file folders and file holders with organizational extras. Record and manage project ...", + "title": "Filing & Folders", + "type": "general", + "url": "https://www.levenger.com/collections/filing-folders" + }, + { + "cite": "https://www.poppin.com › ... › Desk Organization", + "cmpt_rank": 15, + "details": null, + "error": null, + "section": "main", + "serp_rank": 45, + "sub_rank": 0, + "sub_type": null, + "text": "Work happy. Shop our Colorful Filing Systems & Folders now at Poppin.com.", + "title": "Colorful Filing Systems & Folders | Office Supplies", + "type": "general", + "url": "https://www.poppin.com/desktop-organization/filing-folders/" + }, + { + "cite": "https://www.amazon.com › AmazonBasics-File-Folders-...", + "cmpt_rank": 16, + "details": { + "price": "ating: 4.8", + "stock": " 46,992 reviews", + "type": "product" + }, + "error": null, + "section": "main", + "serp_rank": 46, + "sub_rank": 0, + "sub_type": "submenu_product", + "text": "100 manila letter-size file folders ; ideal for 8-1/2 x 11-inch documents; 1/3 cut reinforced tabs in assorted positions (left, middle, right) for easy ...", + "title": "Amazon Basics 1/3-Cut Tab, Assorted Positions ...", + "type": "general", + "url": "https://www.amazon.com/AmazonBasics-File-Folders-Reinforced-Tab/dp/B01LYHE49W" + }, + { + "cite": null, + "cmpt_rank": 17, + "details": { + "heading": "Related searches", + "items": [ + "file folders organizer", + "file folder plastic", + "file folders hanging", + "amazon.com file folders", + "file folders nearby", + "file folders with fasteners" + ], + "type": "text" + }, + "error": null, + "section": "footer", + "serp_rank": 47, + "sub_rank": 0, + "sub_type": "related_searches", + "text": "file folders organizer<|>file folder plastic<|>file folders hanging<|>amazon.com file folders<|>file folders nearby<|>file folders with fasteners", + "title": null, + "type": "searches_related", + "url": null + }, + { + "cite": null, + "cmpt_rank": 18, + "details": { + "items": [ + "Overview", + "Design", + "Types", + "Box" + ], + "type": "panel" + }, + "error": null, + "section": "rhs", + "serp_rank": 48, + "sub_rank": 0, + "sub_type": "panel_rhs", + "text": null, + "title": "Things to know", + "type": "knowledge", + "url": null + } + ] +} diff --git a/tests/fixtures/serps-parser-coverage.json.bz2 b/tests/fixtures/serps-parser-coverage.json.bz2 deleted file mode 100644 index b1e91d59..00000000 Binary files a/tests/fixtures/serps-parser-coverage.json.bz2 and /dev/null differ diff --git a/tests/fixtures/serps-sge-2024.json.bz2 b/tests/fixtures/serps-sge-2024.json.bz2 deleted file mode 100644 index 0cf396c2..00000000 Binary files a/tests/fixtures/serps-sge-2024.json.bz2 and /dev/null differ diff --git a/tests/fixtures/serps-v0.6.7.json.bz2 b/tests/fixtures/serps-v0.6.7.json.bz2 deleted file mode 100644 index 1a9dd954..00000000 Binary files a/tests/fixtures/serps-v0.6.7.json.bz2 and /dev/null differ diff --git a/tests/fixtures/serps-v0.7.2-ads.json.bz2 b/tests/fixtures/serps-v0.7.2-ads.json.bz2 deleted file mode 100644 index 0a00a43a..00000000 Binary files a/tests/fixtures/serps-v0.7.2-ads.json.bz2 and /dev/null differ diff --git a/tests/fixtures/serps-v0.7.2-jobs.json.bz2 b/tests/fixtures/serps-v0.7.2-jobs.json.bz2 deleted file mode 100644 index acc03a1b..00000000 Binary files a/tests/fixtures/serps-v0.7.2-jobs.json.bz2 and /dev/null differ diff --git a/tests/fixtures/serps-v0.7.2-knowledge-subcards.json.bz2 b/tests/fixtures/serps-v0.7.2-knowledge-subcards.json.bz2 deleted file mode 100644 index 04270aa4..00000000 Binary files a/tests/fixtures/serps-v0.7.2-knowledge-subcards.json.bz2 and /dev/null differ diff --git a/tests/fixtures/serps-v0.6.8.json.bz2 b/tests/fixtures/serps.json.bz2 similarity index 55% rename from tests/fixtures/serps-v0.6.8.json.bz2 rename to tests/fixtures/serps.json.bz2 index a33bd69c..2eb85109 100644 Binary files a/tests/fixtures/serps-v0.6.8.json.bz2 and b/tests/fixtures/serps.json.bz2 differ diff --git a/tests/test_ai_overview_legacy_sge.py b/tests/test_ai_overview_legacy_sge.py index ff3b3670..31f4dde3 100644 --- a/tests/test_ai_overview_legacy_sge.py +++ b/tests/test_ai_overview_legacy_sge.py @@ -16,7 +16,7 @@ import WebSearcher as ws -FIXTURE = Path(__file__).parent / "fixtures" / "serps-sge-2024.json.bz2" +FIXTURE = Path(__file__).parent / "fixtures" / "serps.json.bz2" # qry -> whether the overview carries recoverable content (vs a genuine failure) CONTENT_QRYS = { diff --git a/tests/test_parse_serp.py b/tests/test_parse_serp.py index d123a6c7..7ef2fc7d 100644 --- a/tests/test_parse_serp.py +++ b/tests/test_parse_serp.py @@ -14,7 +14,8 @@ # --------------------------------------------------------------------------- FIXTURES_DIR = Path(__file__).parent / "fixtures" -SERPS_PATHS = sorted(FIXTURES_DIR.glob("serps-v*.json.bz2")) +SERPS_PATH = FIXTURES_DIR / "serps.json.bz2" +SERPS_PATHS = [SERPS_PATH] if SERPS_PATH.exists() else [] def load_serps(path: Path) -> list[dict]: diff --git a/tests/test_parser_coverage.py b/tests/test_parser_coverage.py index 4ebba0bb..e94a69c9 100644 --- a/tests/test_parser_coverage.py +++ b/tests/test_parser_coverage.py @@ -1,8 +1,8 @@ """Targeted tests for parser-coverage fixes (plan 024). -Each test loads the curated ``serps-parser-coverage.json.bz2`` fixture, whose -SERPs were selected to exercise specific parser gaps. Tests are added per phase -as the fixes land. +Each test looks up curated SERPs by query in the consolidated ``serps.json.bz2`` +fixture; these queries were selected to exercise specific parser gaps. Tests are +added per phase as the fixes land. """ import bz2 @@ -13,7 +13,7 @@ import WebSearcher as ws -FIXTURE = Path(__file__).parent / "fixtures" / "serps-parser-coverage.json.bz2" +FIXTURE = Path(__file__).parent / "fixtures" / "serps.json.bz2" def _load():