top of page
Task:
The customer required a data processing system with loading into the intermediate storage (AWS S3), and then to permanent data storage (AWS Redshift).
Solution:
-
Determined the optimal amount of resources for AWS Glue
-
Created different kinds of AWS Glue Jobs: based on Apache Spark and Python Shell Jobs using Awswrangler
-
Developed data orchestration using AWS StepFunctions
-
Local testing using Docker
-
Created AWS StepFunctions to automate data manipulation

Project Results
Automated data extraction, conversion and loading system created, tested and handed over to the customer

Backoffice Application

bottom of page