life is too short for a diary
Cross Region AWS Glue Data Catalog access with Glue ETL
AWS Glue job are fully managed ETL service. Glue job uses hive-compatible metastore called Glue Data catalog.
So one way to create dynamic frame in AWS Glue (Pyspark) is
This Glue job reads Glue database defined in same region. As if now, there's no way to access cross-region database in Glue catalog.
However, if the s3 bucket is global, you can directly create dataframe from s3 bucket.
comments powered by Disqus