Aggregate Dataset Data

Use aggregation when the API should filter, group, or project records before returning them. Start with the simple aggregation endpoint; use a pipeline only when the simple form cannot express the calculation.

Simple aggregation

GET/api/v1/data/{provider}/{dataset}/aggregate/

Name	Type	Requirement	Description
provider	String	Required	Dataset provider.
dataset	String	Required	Dataset name.
match	JSON object	Required	Records included in the aggregation.
sort	JSON object	Optional	Sort stage.
limit	Integer	Required	Maximum result size, from 1 through 10,000.
skip	Integer	Optional	Records to skip before limiting.
group	JSON object	Optional	MongoDB-style group stage.
project	JSON object	Optional	Fields included or excluded from the result.

Example: maximum metric value

cURL

curl --get 'https://data.corva.ai/api/v1/data/corva/metrics/aggregate/' \
  --header "Authorization: API ${CORVA_API_KEY}" \
  --data-urlencode 'match={"asset_id":12345,"data.key":"max_dls"}' \
  --data-urlencode 'sort={"timestamp":-1}' \
  --data-urlencode 'limit=100' \
  --data-urlencode 'project={"_id":0,"timestamp":1,"data.key":1,"data.value":1}'

Python

response = requests.get(
    "https://data.corva.ai/api/v1/data/corva/metrics/aggregate/",
    headers=headers,
    params={
        "match": json.dumps({"asset_id": 12345, "data.key": "max_dls"}),
        "sort": json.dumps({"timestamp": -1}),
        "limit": 100,
        "project": json.dumps({
            "_id": 0,
            "timestamp": 1,
            "data.key": 1,
            "data.value": 1,
        }),
    },
    timeout=30,
)
response.raise_for_status()

Complex aggregation pipeline

POST/api/v1/data/{provider}/{dataset}/aggregate/pipeline/

For complex pipelines, prefer POST so the stages are sent as JSON instead of being embedded in a query string.

response = requests.post(
    "https://data.corva.ai/api/v1/data/corva/directional.slide-sheet/aggregate/pipeline/",
    headers=headers,
    json={
        "stages": [
            {"$match": {"asset_id": 12345}},
            {"$limit": 1000},
            {"$project": {"_id": 0, "timestamp": 1, "data": 1}},
        ]
    },
    timeout=30,
)
response.raise_for_status()

caution

Aggregation support and dataset indexes vary. Validate the pipeline with a small limit and review the maintained Data API Reference before using additional operators.