11 of 11 Apache Jobs in the North West

practices Essential Skills & Experience ·Proven experience as a Data Engineer working in a modern cloud data platform ·Strong hands‐on experience with Databricks and Apache Spark ·Experience building and maintaining medallion (Bronze/Silver/Gold) architectures ·Experience extracting and ingesting data using APIs ·Strong SQL skills for data ...

Apache Spark Python AWS Cloud Data Pipelines A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

Apache Spark Python AWS Cloud Data Pipelines A hands-on data engineering role within a large-scale cloud data programme, responsible for building, maintaining, and troubleshooting data pipelines using Apache Spark, PySpark, Apache Airflow, and a broad suite of AWS services. You will apply strong analytical … complex, cloud-based data programme - designing, building, and maintaining data pipelines that process large volumes of data across a modern AWS-native stack. Using Apache Spark and PySpark for distributed data processing, Apache Airflow for orchestration, and a range of AWS services for storage, compute, and analytics ...

patching -Collaborating with Dev & DevOps on infrastructure needs -Identifying improvements to efficiency and service quality -Documenting systems & upholding security standards Your toolkit Linux Ubuntu Apache, PHP, MySQL, PostgreSQL, Nginx, PostfixGit AWS EC2, ECS, Lambda, VPC, Route53, S3RDS, CloudWatch CloudFormation Automation & config Puppet, Ansible, Terraform, scripting langs Python/Bash ...

language used for automation (e.g. Python, JavaScript, TypeScript). · Experience integrating automated tests into CI/CD pipelines, or using orchestration platforms such as Apache Airflow · Practical experience working with real and/or virtual devices (Android, iOS, emulators/simulators) and dealing with the challenges of scale, stability ...

looking for: · 3+ years of professional Python development experience as a Data Platform Engineer, using AWS. · Experience developing and deploying Serverless technologies, provisioning Apache Airflow, and using infrastructure as code such as AWS CloudFormation or Terraform. · Takes a collaborative approach to problem solving and breaking down complex problems, whilst … with software engineering requirements and specifications. · Good interpersonal skills, positive can-do attitude and willing to help other members of the team. · Understanding of Apache Spark, Apache Trino or other big data and machine learning systems. · Exposure to using AI tools to enhance productivity and quality of coding ...

coding skills (any version, but we use Oracle) Experience coding in at least one dynamic scripting language (e.g., Python, Node JS, PowerShell or Apache Groovy) Business reporting intelligence experience (using tools like BIRT, Tableau or Power BI) Knowledge of REST APIs and standard HTTP methods and data migration; either ...

hold active Enhanced DV Clearance (West) Competitive Salary DOE - 6% bonus, 25 days holiday, clearance bonus Experience in Data Pipelines, ETL processing, Data Integration, Apache, SQL/NoSQL Who Are We? Our client is a trusted and growing supplier to the National Security sector, delivering mission-critical solutions that … Engineer Should Have: Active eDV clearance (West) Willingness to work full-time on-site in Manchester when required. Required technical experience in the following: Apache Kafka Apache NiFI SQL and noSQL databases (e.g. MongoDB) ETL processing languages such as Groovy, Python or Java To be Considered: Please either ...

optimise scalable data pipelines within a Databricks Lakehouse environment. You’ll work with both streaming and batch data pipelines using technologies such as Apache Spark Structured Streaming, Delta Live Tables, and Kafka, ensuring data flows efficiently and reliably across the organisation. Working closely with BI, product, and engineering teams …/7 Collaborative office environment with regular social and charity events And Much More!!! DATA ENGINEER – ESSTENTIAL SKILLS Strong experience with Databricks and Apache Spark (PySpark or Scala) Experience building streaming and batch pipelines using Spark Structured Streaming Hands-on experience with Kafka (MSK) and real-time data ingestion ...

delivery of data products. Develop scalable cloud-native solutions using AWS technologies. Build and maintain backend services and infrastructure using Python. Work with Apache Spark and Apache Airflow for data orchestration pipelines Implement infrastructure as code using Terraform and/or AWS CloudFormation . Manage IAM setups … experience focused on AWS-based data platforms. Strong experience with DevOps, cloud engineering and running production data platforms. Hands-on experience provisioning and managing Apache Airflow and Apache Spark . Experience with CI/CD pipelines, automated testing and agile software development practices. Comfortable working with large ...

requirements 3+ years of Python in production environments Strong AWS experience across core services Strong SQL skills for querying, transformation, and optimisation Experience with Apache Spark or serverless architectures Core responsibilities and skills Build and maintain real-time and batch ETL pipelines using Python on large, complex datasets Write … across large data volumes Model and transform data into well-structured, reusable datasets for downstream use Work with distributed data processing tools such as Apache Spark to handle high-volume workloads Improve existing pipelines by identifying bottlenecks and refactoring code and queries Develop and maintain data workflows, scheduling ...