The Big Small BigQuery Guide - Basics Part Two Point Three - Basic SDK operations

SDK operations for a cloud environment ... an interesting concept. Not because code is bad, it's just really weird, y'know? Like wiping while standing up (or sitting down for you maniacs out there).

So far, we’ve covered two ways to use the functionality of BigQuery (here and here) and they have - both of them - given us different perspectives on the BigQuery service. There is now a third and final perspective that we must understand to fully cover the basics of BigQuery: SDKs. Using code and SDKs to operate on BigQuery can be a preference for a lot of people who come from that background. It is also very useful for API integrations with BigQuery, of which there are many.

Introduction

If you are a software developer or, at least, someone who likes to write code, you can more than likely find the way to install and use BigQuery in your programming language right here. I’m not bogging you down with these details, this is going to be a short blog. All you need to run a BigQuery API (as I will in Python) is the query that you are about to run and a few lines of code.

Code

Let’s reuse the code that we used for the “Select All” query from the very first blog. Simply put that query in the query slot and you’ll receive the exact same result.

from google.cloud import bigquery

client = bigquery.Client()

query = """
    SELECT * FROM `bigquery-public-data.google_trends.top_terms` LIMIT 1000
"""
rows = client.query_and_wait(query) .

print("The query data:")
for row in rows:
    print(row)

And that’s what you get. You may manipulate the data in any way you like. This can be useful for things like custom machine learning algorithms and training said algorithms (though you may find there are easier ways to do this in BigQuery itself).

Conclusion

SDKs are useful for a lot of things. Not just in BigQuery, but with most services. The use case for them in BigQuery in particular depends on your creativity. Big, useful concepts like IaC were born from people using this kind of thinking. So, to conclude, I would like to encourage you to be creative and start from a blank canvas. More pearls of wisdom next time.