What is in this dataset, and what would scanning it cost
Use “What is in this dataset, and what would scanning it cost” in Recued. Survey a BigQuery dataset from metadata alone: which tables exist, how large one of them is, whether it is partitioned, and when it last changed.
How it works
Inspect the data fetches, transforms, gates, and output this recipe runs.
Process (29 steps)
project_text
trim
Trim whitespace from setting project id
dataset_text
trim
Trim whitespace from setting dataset id
project_required
template
Generate text from a template
fail: step.project_text is_empty
dataset_required
template
Generate text from a template
fail: step.dataset_text is_empty
table_cap
clamp
Clamp setting table limit between 1 and 50
dataset_raw
recued-core.bigquery.dataset.read
Read dataset on bigquery
dataset_location
default
Apply default
tables_raw
recued-core.bigquery.table.search
Search table on bigquery
table_stubs
default
Apply default
stub_count
count
Count items in table stubs
have_tables
template
Generate text from a template
fail: step.table_stubs is_empty
first_table
default
Apply default
detail_raw
recued-core.bigquery.table.read
Read table on bigquery
defaults
defaults
Apply defaults
bytes_num
to_number
Convert bytes text to a number
rows_num
to_number
Convert rows text to a number
gib_raw
math
Calculate: {{step.bytes_num}} / 1073741824
gib
round
Round gib raw to 2 decimal places
is_large
compare
Check if bytes num is greater than setting large bytes
size_verdict_large
template
Generate text from a template
skip: step.is_large equal false
size_verdict_small
template
Generate text from a template
skip: step.is_large equal true
defaults_2
defaults
Apply defaults
unpartitioned_note
template
Generate text from a template
skip: step.partitioning is_not_null
partitioned_note
template
Generate text from a template
skip: step.partitioning is_null
partition_state
default
Apply default
modified_ms
to_number
Convert detail raw result lastModifiedTime to a number
modified_at
date_format
Format as a date
card
to_summary
Format results as a summary card
table_list
to_table
Format results as a data table
▼
Output
summary
card
table
table_list
Settings
Configurable at install. Defaults shown — change them anytime in Recued.
dataset id
setting
=
project id
setting
=
large bytes
setting
=
1073741824
table limit
setting
=
25
Uses (3 operations)
Pack operation
Read dataset on bigquery
recued-core.bigquery.dataset.read
Pack operation
Search table on bigquery
recued-core.bigquery.table.search
Pack operation
Read table on bigquery
recued-core.bigquery.table.read
Trust & control
What installing this recipe would let it do. Recued grants these permissions at install — review them there before approving.