Skip to content

Customer Data Platform: pass schema as option to PTransform #88

@iht

Description

@iht

Schema is a local file that is not part of the Python package, so this is going to fail in Dataflow probably:

# Get the schema of bigquery output table
with open('./schema/unified_table.json') as schema_file:
output_schema = json.load(schema_file)

The schema should be read when the pipeline is launched and passed as a string to the PTransform.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions