Run Trends API inside a Dagster asset

A Dagster asset is a Python function that POSTs to https://api.trendsapi.ai/api with a Bearer key, then parses body a second time because that field is a JSON string. The same contract covers get_growth, get_time_series, and get_top_trends. On 2026-09-02 the python source last complete week for dagster was 2026-08-24 at score 69.6 and 1,849,133 downloads, down from 100.0 and 2,644,156 the week of 2026-08-17. Peak week in the series is that 2026-08-17 row, not the lagging tip. A weekly cron of three PyPI keywords plus one Google Search keyword stays inside the 100-request free tier. Daily polling of that set does not.

How does a Dagster asset call Trends API?

The HTTP shape is one POST. Mode, source, and keyword sit in the JSON body. Auth is Authorization: Bearer. Dagster adds materialization, retries, and a cron schedule around that call.

POST https://api.trendsapi.ai/api
Authorization: Bearer <api_key>
Content-Type: application/json

{"mode":"get_growth","source":"python","keyword":"dagster","percent_growth":["7D","30D","12M"]}

import json
import os
import urllib.request

from dagster import Definitions, ScheduleDefinition, asset, define_asset_job

API = "https://api.trendsapi.ai/api"


def get_growth(keyword: str, source: str) -> dict:
    payload = {
        "mode": "get_growth",
        "source": source,
        "keyword": keyword,
        "percent_growth": ["7D", "30D", "12M"],
    }
    key = os.environ["TRENDSAPI_API_KEY"]
    req = urllib.request.Request(
        API,
        data=json.dumps(payload).encode("utf-8"),
        headers={
            "Authorization": f"Bearer {key}",
            "Content-Type": "application/json",
        },
        method="POST",
    )
    with urllib.request.urlopen(req, timeout=60) as resp:
        envelope = json.loads(resp.read().decode("utf-8"))
    if envelope.get("statusCode") != 200:
        raise RuntimeError(envelope)
    # body is a JSON string; parse it a second time
    return json.loads(envelope["body"])


@asset
def dagster_pypi_growth() -> dict:
    return get_growth("dagster", "python")


@asset
def airflow_pypi_growth() -> dict:
    return get_growth("apache-airflow", "python")


@asset
def prefect_pypi_growth() -> dict:
    return get_growth("prefect", "python")


@asset
def dagster_search_growth() -> dict:
    return get_growth("dagster", "google search")


watchlist_job = define_asset_job(
    "orchestrator_watchlist",
    selection=[
        dagster_pypi_growth,
        airflow_pypi_growth,
        prefect_pypi_growth,
        dagster_search_growth,
    ],
)
weekly = ScheduleDefinition(job=watchlist_job, cron_schedule="0 10 * * 1")

defs = Definitions(
    assets=[
        dagster_pypi_growth,
        airflow_pypi_growth,
        prefect_pypi_growth,
        dagster_search_growth,
    ],
    jobs=[watchlist_job],
    schedules=[weekly],
)

Schedule wiring follows the Dagster schedule docs. The Prefect equivalent is a @flow with a cron deployment. See Prefect Trends API for that path. The GitHub Actions equivalent is a workflow YAML cron on GitHub Actions Trends API. The Python client without an orchestrator is on Python Trends API.

What did dagster PyPI and npm return on 2026-09-02?

get_growth on source python and keyword dagster used recent_date 2026-08-24. That week scored 69.6 with 1,849,133 downloads. The 7D baseline week 2026-08-17 scored 100.0 with 2,644,156 downloads, growth -30.4 percent, volume growth -30.07 percent. The 14D baseline 2026-08-10 scored 95.1 / 2,516,464, growth -26.81 percent. The 30D baseline 2026-07-27 scored 86.7 / 2,297,369, growth -19.72 percent. The 3M baseline 2026-05-25 scored 64.4 / 1,712,306, growth +8.07 percent. The 12M baseline 2025-08-25 scored 29.9 / 810,284, growth +132.78 percent. YTD vs 2025-12-29 (17.1 / 476,722) was +307.02 percent.

get_time_series on the same source puts the 2026-08-24 week in context. Peak week was 2026-08-17 at 100.0 / 2,644,156. The 2026-08-10 week was 95.1 / 2,516,464. The 2026-08-03 week was 94.5 / 2,500,160. PyPI weekly tips often lag Google weekly sources by about a week, so treat 2026-08-24 as the latest complete python week, not a same-day match for Search week 2026-08-29. The 2026-08-24 drop vs 2.64 million looks like a lagging or incomplete tip, not a product crash. Store recent_date next to the score.

npm is a different registry and a different scale. Keyword dagster on source npm last complete week 2026-08-29 scored 7.8 with 5 downloads. The prior week 2026-08-22 scored 6.2 / 4. get_growth also returned a partial future-dated week 2026-09-05 at 4.7 / 3. Filter get_time_series to dates on or before 2026-09-02 before computing growth. Series peak on npm was 2024-01-13 at 100.0 / 64 downloads. Five weekly npm downloads against 1.85 million PyPI downloads is not a second signal. Skip npm for this keyword, or store it as a registry-mismatch check.

Wikipedia returned 404 no_data for both dagster and Dagster (software). Absence of a Wikipedia series is a finding. Do not invent a page-view row.

How do Prefect, Airflow, and dbt-core compare on the same python source?

Scores are 0-100 per keyword. Do not rank 69.6 against 43.6 as if they share a scale. Compare absolute recent_volume.

Airflow still led the three orchestrators on raw downloads that week (2,143,430 vs 1,849,133 vs 1,679,938) even after the steepest 7D drop. Dagster 12M growth was +132.78 percent vs 2025-08-25 (29.9 / 810,284). Prefect 12M was -0.23 percent vs 43.7 / 1,686,465. Airflow 12M was -42.25 percent vs 35.5 / 3,332,094. dbt-core is not an orchestrator. It is here as a join trap: 6,654,275 downloads at score 22.6 still dwarf all three, and the 7D volume drop to 21,779,634 from the week of 2026-08-17 is the same incomplete-tip pattern. Do not put dbt-core on the orchestrator chart.

All four 2026-08-24 weeks sit well below their own 2026-08-17 volumes. A Monday job that alerts on 7D volume growth below -25 percent would have fired for every row in that table. Gate the alert on recent_date being a complete week, or compare 14D (dagster -26.52 percent volume vs 2,516,464) instead of the lagging tip.

What does Google Search show against download volume?

Search weeks end 2026-08-29, five days after the latest PyPI week. Keyword dagster scored 20.0, 7D growth +5.26 percent vs 19.0 on 2026-08-22. 14D was -35.48 percent vs 31.0 on 2026-08-15. 30D was -57.45 percent vs 47.0 on 2026-08-01. 3M was -64.91 percent vs 57.0 on 2026-05-30. 12M was -61.54 percent vs 52.0 on 2025-08-30. YTD vs 18.0 on 2026-01-03 was +11.11 percent.

prefect Search scored 30.0, 7D growth -6.25 percent vs 32.0. apache airflow scored 18.0, 7D -10.0 percent vs 20.0. github actions scored 12.0, 7D -58.62 percent vs 29.0. Search and PyPI do not share a week ending, and they do not share a scale. Join on keyword plus recent_date, not on score.

YouTube for dagster was 43.0 on 2026-08-29, 7D growth +79.17 percent vs 24.0 on 2026-08-22. 14D was -31.75 percent vs 63.0. 30D was -43.42 percent vs 76.0. 3M and 12M were both -14.0 percent vs 50.0. Video interest moved up on the 7D window while Search only ticked from 19.0 to 20.0. Store both.

Google News for dagster was 3.0 vs 3.0 on the 7D window (0.0 percent). 14D was -66.67 percent vs 9.0. The 30D, 3M, and 12M News windows all baselined at 0.0 and returned growth 999999 with a low-quality warning (over 90 percent zeros). Print the raw 3.0 score. Do not store 999999 as a growth feature.

Source fields for Search are on Google Trends docs.

How many requests does a weekly watchlist use?

Only HTTP 200 responses count against quota. Free tier 100 per month. Starter 5,000. Pro 25,000. Business 100,000.

The job above is 4 get_growth calls. Weekly Monday cron is 4 × 4 = 16 in a 30-day month, plus retries. Two retries on a flaky call still stay under 100. Daily cron is 4 × 30 = 120 before retries, which exceeds free. Add a get_top_trends Google Trends board and the weekly total is 5 calls, about 20 per month.

get_growth with comma-separated sources can 504. Call one source per asset, which is also how Dagster retries stay scoped to a single source.

How should the job fail a run?

Raise after the second parse, not on the HTTP 200. A 200 with inner status error or a 404 no_data is a finding. The Wikipedia dagster 404 is the example: the asset should record data_unavailable and keep the rest of the watchlist. Fail the job only when a required source (here python for dagster) is missing, or when a stored rule fires after a complete week, such as dagster PyPI 14D volume growth below -25 percent (the 2026-08-10 to 2026-08-24 window sat at -26.52 percent).

Do not fire that rule on the 2026-08-24 vs 2026-08-17 7D window until the next python tip arrives. Mixing PyPI week 2026-08-24 with Search week 2026-08-29 without the dates is the usual join bug.

A dated snapshot a Monday job would have caught this week is Dolly Parton vs tropical storm Dolly. Cron without Dagster is on cron Trends API. Airflow DAG wiring is a separate thin page at airflow Trends API.