Cross cloud data analytics – BigQuery Omni for AWS and Azure
Industry research shows that 90 percent of enterprises have a multi-cloud strategy. This is a challenge in terms of integrating and orchestrating data - and for governance. In addition, data silos often arise. A recent remedy is BigQuery Omni, a multi-cloud analytics service that enables data professionals to break down data silos and analyze across different clouds securely and cost-effectively.
Multicloud is the future
With BigQuery Omni, organizations are now able to perform cross-cloud analytics from a single interface across Google Cloud, Amazon Web Services (AWS) and Microsoft Azure. BigQuery Omni will be available to all AWS customers and select Microsoft Azure customers in the fourth quarter.
Data analysts will be able to query data directly from the familiar BigQuery user interface, harnessing the power of BigQuery where the data resides.
Here are some ways BigQuery Omni addresses current challenges with multi-cloud environments:
- Multicloud is here to stay: Enterprises are not consolidating, they are expanding their data stack across clouds. For financial, strategic, and policy reasons customers need data residing in multiple clouds. Support of multicloud has become a minimum requirement for data platforms.
- Multicloud should work seamlessly: Providing a single-pane-of-glass over all data stores empowers a data analyst to extend their ability to drive business impact without learning new skills and shouldn’t need to worry about where the data is stored. Because BigQuery Omni is built using the same APIs as BigQuery, where data is stored (AWS, Azure, or Google Cloud) becomes an implementation detail.
- Consistent security patterns are crucial for enterprises to scale: As more data assets are created, providing the correct level of access can be challenging. Security teams need control over all data access with as much granularity as possible to ensure trust and data synchronization.
- Data quality unlocks innovation: Building a full cross-cloud stack is only valuable if the end user has the right data they need to make a decision. Multiple copies, inconsistent, or out-of-date data all drive poor decisions for analysts.
The Google Cloud Blog quotes BigQuery customer Johnson & Johnson, an early adopter of BigQuery Omni on AWS
We found that BigQuery Omni was significantly faster than other similar applications. We could write back the query results to other cloud storages easily and multi-user and parallel queries had no performance issues in Omni. How we see Omni is that it can be a single pane of glass using which we can connect to various clouds and access the data using, SQL like queries."
Looking ahead to 2022: introducing new features
In 2022, there will be new features such as cross-cloud transfer and external data sources to help analysts manage governed cross-cloud scenarios and workflows through the BigQuery user interface. In 2022, new capabilities will include cross cloud transfer and authorized external tables to help data analysts drive governed, cross-cloud scenarios and workflows all from the BigQuery interface.
Cross cloud transfer helps move the data you need to finish your analysis in Google Cloud and find insights leveraging unique capabilities of BigQuery ML, Looker and Dataflow. Together, these features enable simplified and secure access across different clouds for all your analytics needs. Below is a brief demo of these features relevant to multi-cloud data analysts and scientists.
