Data Catalog

58 tables across 11 categories, available via a readonly ClickHouse account.

The Surf Data Catalog documents all tables available to agents and analysts. Tables are organized into categories and exposed as ClickHouse views in the agent and curated databases.

Connection

Quick Start

-- Top DEX protocols by volume yesterday
SELECT project, sum(amount_usd) AS volume_usd
FROM agent.ethereum_dex_trades
WHERE block_date = today() - 1
GROUP BY project
ORDER BY volume_usd DESC
LIMIT 10

Categories

Query Best Practices

1. Always filter on block_date — partition pruning is the #1 performance lever
2. Use LIMIT liberally during exploration; remove for final aggregation
3. Pre-filter in CTEs before JOINing — ClickHouse puts RIGHT side into hash table
4. Use IN (SELECT ...) instead of JOIN when you only need existence checks
5. Avoid SELECT * on billion-row tables — project only needed columns
6. For ReplacingMergeTree tables, always use FINAL to get deduplicated rows
7. Date range first, then entity filters — this matches partition layout
8. Use toStartOfWeek/toStartOfMonth for time-series rollups, not GROUP BY toString(block_date)
9. When counting unique addresses, uniq() is faster than count(DISTINCT) and equally accurate
10. For cross-table analysis, filter each table independently in CTEs before joining
11. Handle NULLs explicitly — use ifNull(col, default) or coalesce(). ClickHouse 3-valued logic silently drops NULL rows in WHERE filters.
12. FINAL syntax: FROM table_name alias FINAL (not FROM table_name FINAL alias). Place FINAL after the alias.

Entity Linking

Common join patterns across tables:

• token_address -> symbol: JOIN agent.ethereum_erc20 ON contract_address
• condition_id -> market_question: JOIN agent.polymarket_market_details ON condition_id
• market_ticker -> category: Use curated.kalshi_daily_categorized (pre-joined)
• coin -> max_leverage: JOIN agent.hyperliquid_perp_meta ON coin
• project + version -> TVL: JOIN agent.ethereum_tvl_daily ON (project, version, block_date)

Data Freshness

Check freshness for any table:

SELECT max(block_date) FROM agent.<table_name>