All technological notes.
AWS GlueAWS Glue
extract, transform, and load (ETL) serviceGlue Job Bookmarks:
Glue Elastic Views:
“virtual table” (materialized view)Glue DataBrew:
Glue Studio:
ETL jobs in GlueGlue Streaming ETL (built on Apache Spark Structured Streaming):
Kinesis Data Streaming, Kafka, MSK (managed Kafka)Glue Data CatalogGlue Data Catalog
Glue Data Crawler, which is connected to various data sources and writes all of the metadata into the Glue Data CatalogIt is the central of many services, and used behind the scene like Athena, Redshift, EMR.


Parquet format
Convert CSV in the S3 into Parquet format, then using Athena to analyze.
