Experience: 8+ years IT experience, with at least 5+ years of “Data Engineer/Developer”, focused titles/roles,
Data Tools: Hands-on experience with Apache Beam or Informatica.
Testing: Hands-on experience with automated testing activities/outcomes
DevOps: Experience with the processes involving Microservices facilities for artefacts developed
Platform/OS: Mid-range/Unix
Languages: Unix (Shell) Scripting/commands, SQL, Python
File formats: Text, CSV, Parquet, JSON, XML
Scheduling: Jenkins, Airflow
Auto Scripting: Designing/creation of scripts for repetitive activity. E.g. Unix scripting/SQL
Big data: Exposure to Data Lake Concepts, Hive (Schema), HDFS
Source Control Tools: GIT
Delivery Models: Agile, Scrum
Ingestion Design: Create/assess Source/Target data mapping designs
Frameworks: Development and delivery frameworks
Activity Reporting/Repository: Jira, Confluence.
Quality: Accuracy and attention to detail