Data Engineer-Data Platforms-AWS
As a Data Engineer specializing in Data Platforms on AWS at IBM Consulting, you will advise on, develop, and maintain data engineering solutions within the AWS Cloud ecosystem. You will design, build, and operate both batch and real-time data pipelines using AWS services such as EMR, Glue, Glue Catalog, and Kinesis. Your responsibilities include creating data layers on AWS RedShift, Aurora, and DynamoDB and migrating data using AWS DMS. You will develop pipelines for Data Warehouse and Data Lake architectures utilizing AWS Kinesis and Managed Streaming for Apache Kafka. The role involves scheduling and managing data services on the AWS Platform and leveraging open source technologies like Apache Airflow, dbt, and Spark with Python or Scala. Experience with AWS Glue Databrew for data preparation, Lambda functions with Python for data processing, and RedShift Spectrum for analytics tasks is preferred. A Bachelor's degree is required and a Master's degree is preferred.
