/google-cloud-solution-agentic-analytics-spark-knowledge-catalog

name: google-cloud-solution-agentic-analytics-spark-knowledge-catalog
metadata:
  category: MultiProductSolutions
description: >-
  Discovers requirements and generates guidance to design and deploy a governed,
  secure agentic-analytics solution for data that's distributed across Google
  Cloud, other cloud providers, or on-premises. Data that's outside Google Cloud
  (such as data from Databricks, Snowflake, Salesforce, SAP, or Oracle systems)
  is accessed through federation mechanisms such as Apache Iceberg, other
  "zero-copy ETL" methods, or remote query push-down. Use this skill when
  designing an architecture for efficient analytics across large volumes of
  structured and unstructured data that's located in multiple systems and
  environments, including other cloud providers and on-premises.

Agentic analytics across cloud providers and data types

This skill provides a workflow to design and implement a governed, secure pipeline for agentic analytics solution across structured and unstructured data that's distributed across Google Cloud, on-premises systems, and other cloud providers.

Overview of the workflow

The workflow consists of the following phases:

• **Phase 1: Requirements discovery**. Gather detailed requirements related to

the cloud workload or use case that the user needs assistance for.

• **Phase 2: Solution architecture**. Use the requirements that were gathered

in Phase 1 to generate a detailed solution architecture for the cloud workload or use case.

• **Phase 3: Solution validation**. Create a plan to validate the generated

solution, generate validation instructions and scripts, and run the validation.

• **Phase 4: Solution packing and presentation**. Consolidate the generated

content and present the solution.

**Important notes about the workflow**:

• **Strict phase separation**: During Phase 1 (Requirements discovery), when

you ask the user clarifying questions, DON'T recommend, propose, or outline any architectural designs, technical decompositions, cloud services, or component mappings.

• **When you can skip certain phases**: If the user's prompt indicates that a

specific phase or task in this workflow is already completed or approved (e.g., "requirements discovery stage is completed", "product selection is approved", or "architecture is confirmed"), DON'T repeat that phase or task. Instead, skip directly to the requested task (such as generating the technical decomposition, recommending products, or compiling the solution guide).

Phase 1: Requirements discovery and analysis

1. Request the user to describe the functional requirements (business processes, activities, and use cases) of their workload. Ask the user the following questions, one question at a time:

• What are your primary inventory data sources? Are they unstructured

(e.g., PDF flavor recipes, invoices) or structured (e.g., historical sales in Iceberg)?

• Where are these sources hosted? Are they split across AWS S3, Azure

Blob, Google Cloud Storage, or databases like AlloyDB?

• How do you manage and federate metadata across your data

sources within Google Cloud and in external locations (such as other cloud providers)?

• What are your analytical and computational requirements to join, clean,

and run forecast models over large-scale distributed data?

• What types of natural language prompts do your data scientists or

operational agents expect to execute in their agentic IDE (VS Code or Antigravity IDE)? 2. Request the user to describe the non-functional requirements of their workload.

The following are examples of questions you can ask to gather non-functional requirements:

• **Security, privacy, and compliance**: What data privacy rules,

regulatory compliance (e.g., GDPR, HIPAA), or data governance requirements must the system adhere to?

• **Reliability**: What are your uptime, high-availability,

fault-tolerance, and disaster recovery objectives (RTO/RPO)?

• **Performance**: What target query latencies and SLA expectations does

your workload require?

• **Operations**: What operational monitoring metrics do your data

scientists and engineers need?

• **Cost & Sustainability**: Do you have specific budget constraints and

data egress/transfer cost requirements? 3. Ask the user whether the workload currently runs on other cloud providers or on-premises.

• If the user answers "yes", then ask the user to describe the

architecture of the current deployment.

• If the user answer "no", then proceed to the next step.

4. Request the user to describe dependencies, if any, on other workloads, products, or tools. The following are examples of questions that you can ask to get information about the dependencies:

• Do you have any upstream or downstream dependencies on external systems

(e.g., identity providers, data curation platforms, CI/CD pipelines, or active data catalogs)?

• Are there any requirements for your general data-engineering software

delivery lifecycle (e.g., version control, testing, data quality assurance)? Provide the path to a directory or examples of these artifacts. 5. Review the input that the user has provided so far, and check whether there are any ambiguities or contradictions.

If you identify any ambiguities or contradictions in the requirements that the user has provided (e.g., zero-copy vs copying data to a repository), then do the following for each ambiguity or contradiction that you identify:

• Describe the ambiguity or contradiction (e.g., explain why copying data

contradicts the zero-copy requirement and also incurs data-transfer costs).

• Ask the user how they wish t